
miM 


APPLIED LÏNEAR ALGEBRA 


THIRD 

Edition 


V 7 : 


, tn*/ * * y 

“^S*/ f?> ^vl r 




Richard Bronson 
Gabriel b. Costa 










MATRIX METHODS 



The Student Solutions Manual is now available 
online through separate purchase at 
www.elsevierdirect.com/companions/9780123744272 




MATRIX METHODS: 
Applied Linear Algebra 

Third Edition 


Richard Bronson 

Fairleigh Dickinson University 
Teaneck, New Jersey 

Gabriel B. Costa 

United States Military Academy 
West Point , New York 



ELSEVIER 


AMSTERDAM • BOSTON • HEIDELBERG * LONDON 
NEW YORK • OXFORD • PARIS • SAN DIEGO 
SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO 

Academie Press is an imprint of Elsevier 



Academie Press is an imprint of Elsevier 
30 Corporate Drive, Suite 400, Burlington, MA 01803, USA 
525 B Street, Suite 1900, San Diego, California 92101-4495, USA 
84 Theobald’s Road, London WC1X 8RR, UK 

Copyright © 2009, Elsevier Inc. Ail rights reserved. 

No part of this publication may be reproduced or transmitted in any form or by any means, 
electronic or mechanical, including photocopy, recording, or any information storage and 
retrieval System, without permission in writing from the publisher. 

Permissions may be sought directly from Elsevier’s Science & Technology Rights Department 
in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, E-mail: permissions@elsevier.com. 
You may also complété your request online via the Elsevier homepage (http://elsevier.com), by 
selecting “Support & Contact” then “Copyright and Permission” and then “Obtaining Permissions.” 

Library of Congress Cataloging-in-Publication Data 

APPLICATION SUBMITTED 

British Library Cataloguing-in-Publication Data 

A catalogue record for this book is available from the British Library. 

ISBN: 978-0-12-374427-2 


For information on ail Academie Press publications 
visit our Web site at www.elsevierdirect.com 


Printed in the United States of America 
08 09 10 987654321 


Working together to grow 
libraries in developing countries 

www.elsevier.com | www.bookaid.org | www.sabre.org 


ELSEVIER 


BOOK AID 

International 


Sabre Foundation 





To Evy...again. 
R. B. 


To my brother priests...especially Father Frank Maione, 
the parish priest of my youth...and Archbishop Peter 
Feo Gerety, who ordained me a priest. 


G.B.C. 



This page intentionally left blank 




Contents 


Préfacé xi 

About the Authors xiii 
Acknowledgments xv 

1 Matrices 1 

1.1 Basic Concepts 1 

Problems 1.1 3 

1.2 Operations 6 

Problems 1.2 8 

1.3 Matrix Multiplication 9 

Problems 1.3 16 

1.4 Spécial Matrices 19 

Problems 1.4 23 

1.5 Submatrices and Partitioning 29 

Problems 1.5 32 

1.6 Vectors 33 

Problems 1.6 34 

1.7 The Geometry of Vectors 37 

Problems 1.7 41 

2 Simultaneous Linear Equations 43 

2.1 Linear Systems 43 

Problems 2.1 45 

2.2 Solutions by Substitution 50 

Problems 2.2 54 

2.3 Gaussian Elimination 54 

Problems 2.3 62 


Contents 


2.4 Pivoting Strategies 65 



Problems 2.4 

70 

2.5 

Linear Independence 


Problems 2.5 

76 

2.6 

Rank 78 



Problems 2.6 

83 


2.7 Theory of Solutions 84 

Problems 2.7 87 

2.8 Final Comments on Chapter 2 88 

3 The Inverse 93 

3.1 Introduction 93 

Problems 3.1 98 

3.2 Calculating Inverses 101 

Problems 3.2 106 

3.3 Simultaneous Equations 109 

Problems 3.3 111 

3.4 Properties of the Inverse 112 

Problems 3.4 114 

3.5 LU Décomposition 115 

Problems 3.5 121 

3.6 Final Comments on Chapter 3 124 

4 An Introduction to Optimization 127 

4.1 Graphing Inequalities 127 

Problems 4.1 130 

4.2 Modeling with Inequalities 131 

Problems 4.2 133 

4.3 Solving Problems Using Linear Programming 135 

Problems 4.3 140 

4.4 An Introduction to The Simplex Method 140 

Problems 4.4 147 

4.5 Final Comments on Chapter 4 147 

5 Déterminants 149 

5.1 Introduction 149 

Problems 5.1 150 

5.2 Expansion by Cofactors 152 

Problems 5.2 155 

5.3 Properties of Déterminants 157 

Problems 5.3 161 

5.4 Pivotai Condensation 163 

Problems 5.4 166 



Contents 


ix 

5.5 Inversion 167 

Problems 5.5 169 

5.6 Cramer’s Rule 170 

Problems 5.6 173 

5.7 Final Comments on Chapter 5 173 

6 Eigenvalues and Eigenvectors 177 

6.1 Définitions 177 

Problems 6.1 179 

6.2 Eigenvalues 180 

Problems 6.2 183 

6.3 Eigenvectors 184 

Problems 6.3 188 

6.4 Properties of Eigenvalues and Eigenvectors 190 

Problems 6.4 193 

6.5 Linearly Independent Eigenvectors 194 

Problems 6.5 200 

6.6 Power Methods 201 

Problems 6.6 211 

7 Matrix Calculus 213 

7.1 Well-Defined Functions 213 

Problems 7.1 216 

7.2 Cayley-Hamilton Theorem 219 

Problems 7.2 221 

7.3 Polynomials of Matrices-Distinct Eigenvalues 222 

Problems 7.3 226 

7.4 Polynomials of Matrices — General Case 228 

Problems 7.4 232 

7.5 Functions of a Matrix 233 

Problems 7.5 236 

7.6 The Function e Af 238 

Problems 7.6 240 

7.7 Complex Eigenvalues 241 

Problems 7.7 244 

7.8 Properties of e A 245 

Problems 7.8 247 

7.9 Dérivatives of a Matrix 248 

Problems 7.9 253 

7.10 Final Comments on Chapter 7 254 



X 


Contents 


8 Linear Differential Equations 257 

8.1 Fundamental Form 257 

Problems 8.1 261 

8.2 Réduction of an nth Order Equation 263 

Problems 8.2 269 

8.3 Réduction of a System 269 

Problems 8.3 274 

8.4 Solutions of Systems with Constant Coefficients 275 

Problems 8.4 285 

8.5 Solutions of Systems — General Case 286 

Problems 8.5 294 

8.6 Final Comments on Chapter 8 295 

9 Probability and Markov Chains 297 

9.1 Probability: An Informai Approach 297 

Problems 9.1 300 

9.2 Some Laws of Probability 301 

Problems 9.2 304 

9.3 Bernoulli Trials and Combinatorics 305 

Problems 9.3 309 

9.4 Modeling with Markov Chains: An Introduction 310 

Problems 9.4 313 

9.5 Final Comments on Chapter 9 314 

10 Real Inner Products and Least-Square 315 

10.1 Introduction 315 

Problems 10.1 317 

10.2 Orthonormal Vectors 320 

Problems 10.2 325 

10.3 Projections and QR-Decompositions 327 



Problems 10.3 

337 

10.4 

The QR-Algorithm 


Problems 10.4 

343 

10.5 

Least-Squares 

344 


Problems 10.5 

352 


Appendix: A Word on Technology 355 
Answers and Hints to Selected Problems 357 


Index 411 




Préfacé 


It is no secret that matrices are used in many fields. They are naturally présent 
in ail branches of mathematics, as well as, in many engineering and science fields. 
Additionally, this simple but powerful concept is readily applied to many other 
disciplines, such as économies, sociology, political science, nursing and psychology. 

The Matrix is a dynamic construct. New applications of matrices are still 
evolving, and our third édition of Matrix Methods: Applied Linear Algebra 
(previously An Introduction) reflects important changes that hâve transpired since 
the publication of the previous édition. 

In this third édition, we added material on optimization and probability theory. 
Chapter 4 is new and covers an introduction to the simplex method, one of the 
major applied advances in the last half of the twentieth century. Chapter 9 is 
also new and introduces Markov Chains, a primary use of matrices to probability 
applications. To ensure that the book remains appropriate in length for a one 
semester course, we deleted some of the subject matter that is more advanced; 
specifically, chapters on the Jordan Canonical Form and on Spécial Matrices (e.g., 
Hermitian and Unitary Matrices). We also included an Appendix dealing with 
technological support, such as computer algebra Systems. The reader will also find 
that the text contains a considérable “modeling flavor”. 

This édition remains a textbook for the student , not the instructor. It remains 
a book on methodology rather than theory. And, as in ail past éditions, proofs are 
given in the main body of the text only if they are easy to follow and revealing. 

For most of this book, a firm understanding of basic algebra and a smattering 
of trigonometry are the only prerequisites; any references to calculus are few and 
far between. Calculus is required for Chapter 7 and Chapter 8; however, these 
chapters may be omitted with no loss of continuity, should the instructor wish 
to do so. The instructor will also find that he/she can “mix and match” chapters 
depending on the particular course requirements and the needs of the students. 
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Matrices 


1.1 Basic Concepts 

Définition 1 A matrix is a rectangular array of éléments arranged in horizontal 
rows and vertical columns. Thus, 

1 3 5" 

2 0 -lj’ 



and 


4 1 1 
3 2 1 
0 4 2 



( 2 ) 


( 3 ) 


are ail examples of a matrix. 

The matrix given in (1) has two rows and three columns; it is said to hâve order 
(or size) 2x3 (read two by three). By convention, the row index is always given 
first. The matrix in (2) has order 3x3, while that in (3) has order 3x1. The entries 
of a matrix are called éléments. 

In general, a matrix A (matrices will always be designated by uppercase 
boldface letters) of order pxn is given by 



a il 

0 12 

013 

01n 

021 

022 

023 

• * ‘ 02 n 

031 

032 

• 

. 

033 

• 

. 

03 n 

• 

. 

a p\ 

0/?2 

• 

0/?3 

0/77Z 
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which is often abbreviated to [aij] p x n or just [«//]. In this notation, aq represents 
the general element of the matrix and appears in the ith row and the j th column. 
The subscript /, which represents the row, can hâve any value 1 through p, while 
the subscript j , which represents the column, runs 1 through n. Thus, if i — 2 and 
j = 3, aij becomes «23 and désignâtes the element in the second row and third 
column. If i = 1 and j — 5, aq becomes «15 and signifies the element in the first 
row, fifth column. Note again that the row index is always given before the column 
index. 

Any element having its row index equal to its column index is a diagonal 
element. Thus, the diagonal éléments of a matrix are the éléments in the 1—1 
position, 2—2 position, 3—3 position, and so on, for as many éléments of this type 
that exist. Matrix (1) has 1 and 0 as its diagonal éléments, while matrix (2) has 4, 
2, and 2 as its diagonal éléments. 

If the matrix has as many rows as columns, p — n, it is called a square matrix ; 
in general it is written as 


an 

a \2 

a\3 

• * * a \ n 


an 

ail 

a23 

• * * a 2 n 


an 

• 

• 

an 

a33 

a3n 

(5) 

O-nl 

a n2 

an3 

a,in_ 



In this case, the éléments «11, «22, <233, • • • , a nn lie on and form the main (or 
principal) diagonal. 

It should be noted that the éléments of a matrix need not be numbers; they 
can be, and quite often arise physically as, functions, operators or, as we shall see 
later, matrices themselves. Hence, 


f ( t 2 + 1 )dt t 2 a/3 1 2 

9 

sin 0 cos G 

/A • r\ 

J 0 


— cos G sin 0 


and 

r 2 -1 

X X 

d 

e x — ln x 

dx 

_5 x T 2 _ 

are good examples of matrices. Finally, it must be noted that a matrix is an 
entity unto itself; it is not a number. If the reader is familiar with déterminants, 
he will undoubtedly recognize the similarity in form between the two. Warn- 
ing : the similarity ends there. Whereas a déterminant (see Chapter 5) can be 
evaluated to yield a number, a matrix cannot. A matrix is a rectangular array, 
period. 
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Problems 1.1 


1. Détermine the orders of the following matrices: 


A = 


C = 


3 1-247 

2 5-657 

0 3 12 0 

-3 -5 2 2 2 


1 2 3 4' 

5 6-78 

10 11 12 12 



"1 

2 

3" 

, B = 

0 

0 

0 


1 

3 

2 


E = 


D = 


H = 


"1 1 

2 3 

2 3 

.3 5 

'0 0 ' 
0 0 


1 

4 

5 

6 J 


F = 


1 ' 

5 

10 

0 

-4 


0 


t-2 


6 1 5 

t -|- 2 

3 1 

1 2 

2t — 3 

-5 r 2 

2 1 3 4 5 6 7 3 1 2 


"V3Ï3 

-505" 

G = 

2l T 

18 

46.3 

1.043 


2 V 5 

-V5 


, J = [1 5 -30]. 


2. Find, if they exist, the éléments in the 1—3 and the 2—1 positions for each of 
the matrices defined in Problem 1. 


3. Find, if they exist, «23, «32, &3i, *32, cil, d 2 2,ci3, g22, 823, and /z 32 for the 

matrices defined in Problem 1. 


4. Construct the 2x2 matrix A having «/,• = (— 1)* + K 

5. Construct the 3x3 matrix A having «// = i/j. 

6. Construct the n x n matrix B having b,j = n — i — j. What will this matrix be 

when specialized to the 3x3 case? 

7. Construct the 2x4 matrix C having 




when i = 1, 
when i — 2. 


8. Construct the 3x4 matrix D having 



when i > j , 
when i = j , 
when i < j. 


9. Express the following times as matrices: (a) A quarter after nine in the morn- 
ing. (b) Noon. (c) One thirty in the afternoon. (d) A quarter after nine in the 
evening. 


10. Express the following dates as matrices: 

(a) July 4, 1776 (b) December 7, 1941 

(c) April 23, 1809 (d) October 31, 1688 
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11. A gasoline station currently has in inventory 950 gallons of regular unleaded 
gasoline, 1253 gallons of premium, and 98 gallons of super. Express this 
inventory as a matrix. 

12. Store 1 of a three store chain has 3 refrigerators, 5 stoves, 3 washing machines, 
and 4 dryers in stock. Store 2 has in stock no refrigerators, 2 stoves, 9 washing 
machines, and 5 dryers, while store 3 has in stock 4 refrigerators, 2 stoves, and 
no washing machines or dryers. Présent the inventory of the entire chain as a 
matrix. 

13. The number of damaged items delivered by the SleepTight Mattress Company 
from its various plants during the past year is given by the matrix 


80 

12 

16 

50 

40 

16 

90 

10 

50 


The rows pertain to its three plants in Michigan, Texas, and Utah. The columns 
pertain to its regular model, its firm model, and its extra-firm model, respec- 
tively. The company ’s goal for next year to is to reduce by 10% the number 
of damaged regular maîtresses shipped by each plant, to reduce by 20% the 
number of damaged firm mattresses shipped by its Texas plant, to reduce by 
30% the number of damaged extra-firm mattresses shipped by its Utah plant, 
and to keep ail other entries the same as last year. What will next year’s matrix 
be if ail goals are realized? 

14. A person purchased 100 shares of AT&T at $27 per share, 150 shares of 
Exxon at $45 per share, 50 shares of IBM at $116 per share, and 500 shares of 
PanAm at $2 per share. The current price of each stock is $29, $41, $116, and 
$3, respectively. Represent in a matrix ail the relevant information regarding 
this person’s portfolio. 

15. On January 1, a person buys three certifiâtes of deposit from different insti- 
tutions, ail maturing in one year. The first is for $1000 at 7%, the second is 
for $2000 at 7.5%, and the third is for $3000 at 7.25%. Ail interest rates are 
effective on an annual basis. 

(a) Represent in a matrix ail the relevant information regarding this person’s 
holdings. 

(b) What will the matrix be one year later if each certificate of deposit is 
renewed for the current face amount and accrued interest at rates one 
half a percent higher than the présent? 

16. (Markov Chains, see Chapter 9) A finite Markov chain is a set of objects, 
a set of consecutive time periods, and a finite set of different States such 
that 

(i) during any given time period, each object is in only State (although 
different objects can be in different States), and 
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(ii) the probability that an object will move from one State to another State 
(or remain in the same State) over a time period dépends only on the 
beginning and ending States. 

A Markov chain can be represented by a matrix P = [ /?,/] where pij represents 
the probability of an object moving from State i to State j in one time period. 
Such a matrix is called a transition matrix. 

Construct a transition matrix for the following Markov chain: Census fig- 
ures show a population shift away from a large mid-western metropolitan 
city to its suburbs. Each year, 5% of ail families living in the city move to 
the suburbs while during the same time period only 1 % of those living in the 
suburbs move into the city. Hint : Take State 1 to represent families living in 
the city, State 2 to represent families living in the suburbs, and one time period 
to equal a year. 

17 . Construct a transition matrix for the following Markov chain: Every four 
years, voters in a New England town elect a new mayor because a town 
ordinance prohibits mayors from succeeding themselves. Past data indicate 
that a Démocratie mayor is succeeded by another Democrat 30% of the time 
and by a Republican 70% of the time. A Republican mayor, however, is 
succeeded by another Republican 60% of the time and by a Democrat 40% 
of the time. Hint : Take State 1 to represent a Republican mayor in office, State 

2 to represent a Démocratie mayor in office, and one time period to be four 
years. 

18 . Construct a transition matrix for the following Markov chain: The apple 
harvest in New York orchards is classified as poor, average, or good. His- 
torical data indicates that if the harvest is poor one year then there is a 40% 
chance of having a good harvest the next year, a 50% chance of having an aver- 
age harvest, and a 10% chance of having another poor harvest. If a harvest 
is average one year, the chance of a poor, average, or good harvest the next 
year is 20%, 60%, and 20%, respectively. If a harvest is good, then the chance 
of a poor, average, or good harvest the next year is 25%, 65%, and 10%, 
respectively. Hint : Take State 1 to be a poor harvest, State 2 to be an average 
harvest, State 3 to be a good harvest, and one time period to equal one year. 

19 . Construct a transition matrix for the following Markov chain. Brand X and 
brand Y control the majority of the soap powder market in a particular région, 
and each has promoted its own product extensively. As a resuit of past adver- 
tising campaigns, it is known that over a two year period of time 10% of 
brand Y customers change to brand X and 25% of ail other customers change 
to brand X. Furthermore, 15% of brand X customers change to brand Y and 
30% of ail other customers change to brand Y. The major brands also lose cus- 
tomers to smaller competitors, with 5% of brand X customers switching to a 
minor brand during a two year time period and 2 % of brand Y customers doing 
likewise. Ail other customers remain loyal to their past brand of soap powder. 
Hint : Take State 1 to be a brand X customer, State 2 a brand Y customer, State 

3 another brand customer, and one time period to be two years. 
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1.2 Operations 

The simplest relationship between two matrices is equality. Intuitively one feels 
that two matrices should be equal if their corresponding éléments are equal. This 
is the case, providing the matrices are of the same order. 


Définition 1 Two matrices A = [aij] pxn and B = [bij] pxn are equal if they hâve 
the same order and if a/j = b /y (i = 1, 2, 3, y = 1, 2, 3, ... , n). Thus, the 
equality 


5x + 2y 


"7" 

x — 3 y 


1 


implies that 5x -\-2y — 1 and x — 3 y = 1. 


The intuitive définition for matrix addition is also the correct one. 


Définition 2 If A = [aij] and B = [bjj] are both of order p x n, then A + B is 
a p x n matrix C = [c/y] where c*y — aij + bÿ ( i = 1, 2, 3, . . . , p; j = 1, 2, 3, . . . , n). 
Thus, 


5 

1 


-6 

3 


5 + (—6) 

1 + 3 


"-1 

4 

7 

3 

+ 

2 

-1 

— 

7 + 2 

3 + (—1) 

— 

9 

2 

2 

-1 


4 

1 


(-2) + 4 

(-D + 1 


2 

0 


and 


but the matrices 


■? 2 

5" 

+ 

"1 

-6" 

_3 ? 

0 _ 

t 

—t 


t 2 + 1 - 1 " . 

4 t -t ’ 


5 

0 






—6 

2] 

-1 

0 

and 

1 

i 

2 

1 


i 


cannot be added since they are not of the same order. 


It is not difficult to show that the addition of matrices is both commutative and 
associative: that is, if A, B, C represent matrices of the same order, then 

(Al) A + B = B + A, 

(A2) A + (B + C) = (A + B) + C. 

We define a zéro matrix 0 to be a matrix consisting of only zéro éléments. Zéro 
matrices of every order exist, and when one has the same order as another matrix 
A, we then hâve the additional property 


(A3) A + 0 = A. 
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Subtraction of matrices is defined in a manner analogous to addition: the 
orders of the matrices involved must be identical and the operation is performed 
elementwise. 

Thus, 


— 1 

Ui 

H-* 

1 


'6 -1" 


"-1 2" 

[-3 2_ 


-P- 

1 

h- 1 ‘ 

1 


-1 3_ 


Another simple operation is that of multiplying a scalar times a matrix. Intu- 
ition guides one to perform the operation elementwise, and once again intuition 
is correct. Thus, for example, 


1 

2 " 


7 

14" 

and t 

"1 

0" 


t 

0 ' 

-3 

4 


-21 

28 

3 

2 


_3î 

2 1 _ 


Définition 3 If A = [atj] is a p x n matrix and if À is a scalar, then À A is a p x n 
matrix B = [bjj] where b ij = Xaij{i = 1, 2, 3, . j = 1, 2, 3, ... , n). 

Example 1 Find 5A — if 


Solution 




and 




5A — jB = 5 


"4 

r 

1 

" 6 

-20' 

0 

3 

2 

18 

8 


"20 

5' 


'3 

-10“ 


" 17 

15" 

0 

15 


9 

4 


-9 

11 


It is not difficult to show that if Ai and À 2 are scalars, and if A and B are matrices 
of identical order, then 

(51) ÀiA = Aài, 

(52) Ài(A + B) = MA + ÀiB, 

(53) (Ài + À2)A = ÀiA + ^2 A, 

(54) Ài(À2A) = (ÀiÀ2)A. 

The reader is cautioned that there is no such operation as matrix division. We 
will, however, define a somewhat analogous operation, namely matrix inversion, in 
Chapter 3. 
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Problems 1.2 

In Problems 1 through 26, let 


A = 


1 2 ' 
3 4 


B = 


D = 


5 6 
7 8 


C = 


1 0 ' 
3 -3 


3 

-1 

3 

1 

2 

-2 

, E = 

"-2 

0 

5 

2 

-2 

-3 

, F = 

1 

1 

O H— ^ O 

O O 

1 

(N 

1 

6 


' -n 
i 

1 


— 1 
(N 

(N 
1 


1. Find 2A. 


2. Find — 5A. 


3. Find 3D. 


4. 

Find 10E. 

5. 

Find -F. 


6. 

Find A + B. 

7. 

Find C + A. 

8. 

Find D 

+ E. 


9. 

Find D + F. 

10. 

Find A + D. 

11. 

Find A 

-B. 


12. 

Find C - A. 

13. 

Find D - E. 

14. 

Find D 

-F. 


15. 

Find 2A + 3B. 

16. 

Find 3A — 2C. 

17. 

Find 0.1A + 0. 

,2C. 

18. 

Find — 2E + F. 

19. 

Find X if A + X = 

B. 


20. 

Find Y if 2B + 

Y = C. 

21. 

Find X if 3D - X = 

= E. 


22. 

Find Y if E 

— 2Y = F. 

23. 

Find R if 4A + 5R 

= 10C. 


24. 

Find S if 3F 

- 2S = D. 

25. 

Verify directly that 

(A + B) + C = 

A + 0 

B + C). 



26. 

Verify directly that 

A(A + 

B) = ÀA + ÀB. 




27. 

Find 6A - SB if 








A = 


O 2 26 - ï 
4 1/6 


and B = 


' 6 2 - 1 6 
3/6 6 3 + 26 + 1 


28. Prove Property (Al). 29. Prove Property (A3). 

30. Prove Property (S2). 31. Prove Property (S3). 

32. (a) Mr. Jones owns 200 shares of IBM and 150 shares of AT&T. Détermine a 
portfolio matrix that reflects Mr. Jones’ holdings. 


(b) Over the next year, Mr. Jones triples his holdings in each company. What 
is his new portfolio matrix? 

(c) The following year Mr. Jones lists changes in his portfolio as [—50 100]. 

What is his new portfolio matrix? 
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33. The inventory of an appliance store can be given by a 1 x 4 matrix in which 
the first entry represents the number of télévision sets, the second entry the 
number of air conditioners, the third entry the number of refrigerators, and 
the fourth entry the number of dishwashers. 

(a) Détermine the inventory given on January 1 by [15 2 8 6]. 

(b) January sales are given by [4 0 2 3] . What is the inventory matrix on 
February 1? 

(c) February sales are given by [5 0 3 3], and new stock added in 
February is given by [3 2 7 8]. What is the inventory matrix on 
March 1? 

34. The daily gasoline supply of a local service station is given by a 1 x 3 matrix 
in which the first entry represents gallons of regular, the second entry gallons 
of premium, and the third entry gallons of super. 

(a) Détermine the supply of gasoline at the close of business on Monday 
given by [14,000 8,000 6,000]. 

(b) Tuesday’s sales are given by [3,500 2,000 1,500]. What is the inventory 
matrix at day’s end? 

(c) Wednesday’s sales are given by [5,000 1,500 1,200]. In addition, the 
station received a delivery of 30,000 gallons of regular, 10,000 gallons of 
premium, but no super. What is the inventory at day’s end? 

35. On a recent shopping trip Mary purchased 6 oranges, a dozen grapefruits, 
8 apples, and 3 lemons. John purchased 9 oranges, 2 grapefruits, and 6 
apples. Express each of their purchases as 1 x 4 matrices. What is the physical 
significance of the sum of these matrices? 


1.3 Matrix Multiplication 

Matrix multiplication is the first operation we encounter where our intuition fails. 
First, two matrices are not multiplied together elementwise. Secondly, it is not 
always possible to multiply matrices of the same order while it is possible to mul- 
tiply certain matrices of different orders. Thirdly, if A and B are two matrices for 
which multiplication is defined, it is generally not the case that AB = BA; that is, 
matrix multiplication is not a commutative operation. There are other properties 
of matrix multiplication, besides the three mentioned that defy our intuition, and 
we shall illustrate them shortly. We begin by determining which matrices can be 
multiplied. 

Rule 1 The product of two matrices AB is defined if the number of columns 
of A equals the number of rows of B. 
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Thus, if A and B are given by 



6 1 

-1 2 



and 



-1 0 

3 2 

4 1 


1 0 

-2 1 , 

1 0 



then the product AB is defined since A has three columns and B has three rows. 
The product BA, however, is not defined since B has four columns while A 
has only two rows. 

When the product is written AB, A is said to premultiply B while B is said to 
postmultiply A. 


Rule 2 If the product AB is defined, then the résultant matrix will hâve the same 
number of rows as A and the same number of columns as B. 

Thus, the product AB, where A and B are given in (6), will hâve two rows and 
four columns since A has two rows and B has four columns. 

An easy method of remembering these two rules is the following: write the 
orders of the matrices on paper in the sequence in which the multiplication is to 
be carried out; that is, if AB is to be found where A has order 2x3 and B has 
order 3x4 , write 


(2 x 3) (3 x 4) (7) 

V r 

If the two adjacent numbers (indicated in (7) by the curved arrow) are both equal 
(in the case they are both three), the multiplication is defined. The order of the 
product matrix is obtained by canceling the adjacent numbers and using the two 
remaining numbers. Thus in (7), we cancel the adjacent 3’s and are left with 2x4 , 
which in this case, is the order of AB. 

As a further example, consider the case where A is 4 x 3 matrix while B is 
a 3 x 5 matrix. The product AB is defined since, in the notation (4 x 5)(3 x J), 
the adjacent numbers denoted by the curved arrow are equal. The product will 
be a 4x5 matrix. The product BA however is not defined since in the notation 
(3 x 5) (4 x 3) the adjacent numbers are not equal. In general, one may sche- 
maticaîly State the method as 

(k x n)(n x p) — (k x p). 

Rule 3 If the product AB = C is defined, where C is denoted by [c/ 7 ], then the 
element cij is obtained by multiplying the éléments in ith row of A by the corre- 
spondis éléments in the yth column of B and adding. Thus, if A has order k x n, 
and B has order n x p, and 


an a \2 • • • a\n 


b\\ b\2 b\ p 


CH c\2 ••• C\p 

Ü2l Ü22 • • • a2n 


Z?21 b22 • • • b2 p 



C21 C22 • • • C2p 

• • • 

_a k \ a k 2 ••• a kn _ 


_bnl b n 2 * * * byip _ 


• • • 

• • • 

Ski c k 2 * * * c kp _ 
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then c h is obtained by multiplying the éléments in the first row of A by the 
corresponding éléments in the first column of B and adding; hence, 


en = 011^11 + anb 2 i H h ai n b n 


The element ci2 is found by multiplying the éléments in the first row of A by the 
corresponding éléments in the second column of B and adding; hence. 


c 12 = a llb\2 + «12^22 H h ai n b n 2- 


The element Ck p is obtained by multiplying the éléments in the kth row of A by 
the corresponding éléments in the pth column of B and adding; hence, 


Ckp — Qk\b\p T CLk2b 2 p T * * * T ^knbnp- 


Example 1 Find AB and BA if 



1 2 
4 5 


3 

6 


and B = 


-7 

9 

0 


-8 
10 . 
-11 


Solution 

-7 -8" 

9 10 

0 -11 

1( — 7) + 2(9) + 3(0) 1 ( — 8) + 2(10) + 3(— 11) 
4(— 7) + 5(9) + 6(0) 4(— 8) + 5(10) + 6(— 11) 


AB = 


1 2 3' 

4 5 6 


-7 + 18 + 0 

-8 + 20 - 33 


11 

-21 

-28 + 45 + 0 

-32 + 50-66 


_ 17 

—48+ 


BA = 



-8 

10 

-11 


1 2 3 
4 5 6 


(-7)1 + (-8)4 
9(1) + 10(4) 
0(1) + (-11)4 


(-7)2 + (-8)5 
9(2) + 10(5) 
0(2) + (-11)5 


(-7)3 + (-8)6 
9(3) + 10(6) 
0(3) + (-11)6 


7-32 

-14 - 40 

-21 - 48 


'-39 

-54 

-69" 

9 + 40 

18 + 50 

27 + 60 

— 

49 

68 

87 

0-44 

0-55 

0-66 


-44 

-55 

-66 


The preceding three rules can be incorporated into the following formai 
définition: 
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Définition 1 If A = [a/j] is a k x n matrix and B = \bif\ is an n x p matrix, 
then the product AB is defined to be a k x p matrix C = [q 7 ] where 
Cij = Ya = 1 a il b lj = a nhj + a i2 b 2j + • • • + a in b nj (i = 1,2,...,*; 7 = 1,2,...,/?). 


Example 2 Find AB if 



2 

r 






-1 

0 

and B = 

"3 

1 

5 

1 

- 1 " 

0 

4 

-2 

3 

î 







Solution 


AB = 



1 

0 

1 


3 15-1 

4-2 1 0 


2 ( 3 ) + 1 ( 4 ) 2 ( 1 ) + 1 (— 2 ) 

- 1 ( 3 ) + 0 ( 4 ) — 1 ( 1 ) + 0 (— 2 ) 
3 ( 3 ) + 1 ( 4 ) 3 ( 1 ) + 1 (— 2 ) 


2 ( 5 ) + 1 ( 1 ) 2 (— 1 ) + 1 ( 0 ) 

- 1 ( 5 ) + 0 ( 1 ) — 1 (— 1 ) + 0 ( 0 ) 
3 ( 5 ) + 1 ( 1 ) 3 (— 1 ) + 1 ( 0 ) 


10 

0 

11 

-2 

-3 

-1 

-5 

1 

13 

1 

16 

-3 


Note that in this example the product BA is not defined. ■ 


Example 3 Find AB and BA if 



and 



0 

2 • 


Solution 


AB = 




"4 0 " 


' 2 ( 4 ) + 1 ( 1 ) 

2 ( 0 ) + 1 ( 2 )' 


9 2 ] 

• 


1 2 


- 1 ( 4 ) + 3 ( 1 ) 

— 1 ( 0 ) + 3 ( 2 )_ 


-1 6 _|’ 


BA = 


4 

1 


0 

2 


2 r 


4 ( 2 ) + 0 (— 1 ) 

4 ( 1 ) + 0 ( 3 )' 


"8 4 " 

1 3 


. 1 ( 2 ) + 2 (— 1 ) 

KD + 2 ( 3 )_ 


0 7 


This, therefore, is an example where both products AB and BA are defined but 
unequal. ■ 


Example 4 Find AB and BA if 



3 1 
0 4 


and B = 


'1 1 
0 2 



1.3 Matrix Multiplication 

Solution 




This, therefore, is an example where both products AB and BA are defined and 
equal. ■ 

In general, it can be shown that matrix multiplication has the following 
properties: 

(Ml) A(BC) = (AB)C (Associative Law) 

(M2) A (B + C) = AB + AC (Left Distributive Law) 

(M3) (B + C) A = BA + CA (Right Distributive Law) 

providing that the matrices A, B, C hâve the correct order so that the above 
multiplications and additions are defined. The one basic property that matrix 
multiplication does not possess is commutativity; that is, in general, AB does not 
equal B A (see Example 3). We hasten to add, however, that while matrices in 
general do not commute, it may very well be the case that, given two particular 
matrices, they do commute as can be seen from Example 4. 

Commutativity is not the only property that matrix multiplication lacks. We 
know from our expériences with real numbers that if the product xy = 0, then 
either x — 0 or y = 0 or both are zéro. Matrices do not possess this property as the 
following example shows: 

Example 5 Find AB if 



and B = 




Solution 


4 2] r 3 -4] _ [4(3) + 2(— 6) 4(— 4) + 2(8)' 
2 X J [~ 6 8j — [2(3) 4- 1 ( — 6) 2(— 4) + 1(8) 


Thus, even though neither A nor B is zéro, their product is zéro. ■ 
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One final “unfortunate” property of matrix multiplication is that the équation 
AB = AC does not imply B = C. 


Example 6 Find AB and AC if 




B = 


1 1 ' 
2 1 




Solution 


AB = 



4 2 
2 1 

4 2 
2 1 


1 

2 

2 

0 



4(1) + 2(2) 

4(1) + 2(1)' 


"8 6" 


2(1) + 1(2) 

2(1) + 1(1) 


4 3 


2" 


'4(2) + 2(0) 

4(2) + 2(— 1)~ 


"8 6" 

1 


.2(2) + 1(0) 

2(2) + 1(-1)_ 


4 3 


Thus, cancellation is not a valid operation in the matrix algebra. ■ 


The reader has no doubt wondered why this seemingly complicated procedure 
for matrix multiplication has been introduced when the more obvious methods 
of multiplying matrices termwise could be used. The answer lies in Systems of 
simultaneous linear équations. Consider the set of simultaneous linear équations 
given by 


5x — 3y + 2z = 14, 

* + y - 4z = -7, (8) 

Ix —3 z = 1. 

This System can easily be solved by the method of substitution. Matrix algebra, 
however, will give us an entirely new method for obtaining the solution. 

Consider the matrix équation 

Ax = b (9) 

where 



"5-3 2 




“ 14“ 

A = 

1 1 -4 

, x = 

J 

, and b = 

-7 


7 0-3 


Z 


1 


Here A, called the coefficient matrix , is simply the matrix whose éléments are the 
coefficients of the unknowns jc, y, z in (8). (Note that we hâve been very care- 
ful to put ail the x coefficients in the first column, ail the y coefficients in the 
second column, and ail the z coefficients in the third column. The zéro in the 
(3, 2) entry appears because the y coefficient in the third équation of System (8) 
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is zéro.) x and b are obtained in the obvious manner. One note of warning: there 
is a basic différence between the unknown matrix x in (9) and the unknown vari- 
able x. The reader should be especially careful not to confuse their respective 
identities. 

Now using our définition of matrix multiplication, we hâve that 



"5 

-3 

2 


x 


"(5)0) + (-3)00 + (2)(z) _ 

Ax = 

1 

1 

-4 


J 

— 

(DO) + (1)00 + (— 4)(z) 


7 

0 

-3 


z 


(7)0) + (0)00 + (-3)0) 


5x — 3 y + 2 z 
x + y — 4z 
Ix — 3z 


14 

-7 . 
1 



Using the définition of matrix equality, we see that (10) is precisely System (8). 
Thus (9) is an alternate way of representing the original System. It should corne 
as no surprise, therefore, that by redefining the matrices A, x, b, appropriately, we 
can represent any System of simultaneous linear équations by the matrix équation 
Ax = b. 


Example 7 Put the following System into matrix form: 


v— y + z + w — 5, 

2x -h y - z =4, 
3x + 2y + 2 w = 0, 
x — 2y + 3z + 4w = — 1. 

Solution Define 


"1 

-1 

1 

1 


X 


5" 

2 

3 

1 

2 

1 

O 

0 

2 

, x = 

y 

Z 

, b = 

4 

0 

1 

-2 

3 

4 


w 


-1 


The original System is then équivalent to the matrix System Ax = b. ■ 

Unfortunately, we are not y et in a position to solve Systems that are in matrix 
form Ax = b. One method of solution dépends upon the operation of inversion, 
and we must postpone a discussion of it until the inverse has been defined. For 
the présent, however, we hope that the reader will be content with the knowl- 
edge that matrix multiplication, as we hâve defined it, does serve some useful 
purpose. 
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Problems 1.3 

1. The order of A is 2 x 4, the order of B is 4 x 2, the order of C is 4 x 1, the 
order of D is 1 x 2, and the order of E is 4 x 4. Find the orders of 


(a) AB, 
(e) CD, 

(i) ABC, 


(b) BA, 
(f) AE, 

(j) DAE, 


(c) AC, 

(g) eb, 

(k) EBA, 


(d) CA, 
(h) EA, 

(1) EECD. 


In Problems 2 through 19, let 


A = 


1 2 ' 
3 4 


B = 


'5 6' 
7 8 


C = 


1 0 1 ' 
3-2 1 



1 

1 


~-2 2 1 


0 

1 2 

D = 

-1 

2 

2 

-2 

, E = 

0 -2 -1 

1 0 1 

, F = 

-1 

1 

-1 0 
2 3 


X = [1 - 2], Y = [1 2 1] 


2. Find AB. 

3. FindBA. 

4. Find AC. 

5. Find BC. 

6. Find CB. 

7. Find XA. 

8. Find XB. 

9. Find XC. 

10. FindAX. 

11. Find CD. 

12. Find DC. 

13. Find YD. 

14. FindYC. 

15. Find DX. 

16. Find XD. 

17. Find EF. 

18. Find FE. 

19. Find YF. 


20. Find AB if 


A = 


'2 6 
3 9 


and B = 


3 -6 
-1 2 


Note that AB = 0 but neither A nor B equals the zéro matrix. 
21. Find AB and CB if 


A = 


'3 2' 
1 0 


B = 


'2 4 
1 2 


C = 


1 6 ' 
3 -4 


Thus show that AB = CB but A ^ C 
22. Compute the product 



1.3 Matrix Multiplication 


23. Compute the product 


24. Compute the product 


25. Compute the product 


"1 0 -1 



3 1 1 


J 

1 3 0 


Z 


an au 


X 

_ a 21 U22_ 


_y_ 


b\\ b \2 ^13 

Z?21 k>22 t>23 


-i 

c\T 

i 


-1 

- 

CO 1 
1 


26. Evaluate the expression A 2 — 4A — 51 for the matrix* 


A = 


'1 2 ' 
4 3 


27. Evaluate the expression (A — I) ( A + 21) for the matrix* 


A = 


3 5' 
-2 4 


28. Evaluate the expression (I — A) (A 2 — I) for the matrix* 



-1 1 

-2 1 . 

0 1 


29. Verify property (Ml) for 




0 

1 41’ 



30. Prove Property (M2). 

31. Prove Property (M3). 

32. Put the following System of équations into matrix form: 

2x + 3 y = 10, 

4x — 5y — 11. 


*1 is defined in Section 1.4 
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33. Put the following System of équations into matrix form: 

x + z + y = 2, 

3z + 2x + y = 4, 
y + x = 0. 

34. Put the following System of équations into matrix form: 

5x + 3y + 2z + 4u; = 5, 
x + y 4- w = 0, 

3x + 2y + 2z = —3, 
x + y + 2z 4- 3 tu = 4. 


35. The price schedule for a Chicago to Los Angeles flight is given by P = 
[200 350 500], where the matrix éléments pertain, respectively, to coach 
tickets, business-class tickets, and first-class tickets. The number of tickets 
purchased in each category for a particular flight is given by 



130 
20 . 
10 


Compute the products (a) PN, and (b) NP, and détermine their significance. 

36. The closing priées of a person’s portfolio during the past week are given by 
the matrix 



40 j 

40 1 

3^ 

3- 

o2 

io| 


41 41 

4 3 1 

10 9g 


where the columns pertain to the days of the week, Monday through Friday, 
and the rows pertain to the priées of Orchard Fruits, Lion Airways, and 
Arrow Oil. The person’s holdings in each of these companies are given by 
the matrix H = [100 500 400]. Compute the products (a) HP, and (b) PH, 
and détermine their significance. 

37. The time requirements for a company to produce three products is given by 
the matrix 



0.2 

0.5 

0.4 

1.2 

2.3 

0.7 

0.8 

3.1 

1.2 


where the rows pertain to lamp bases, cabinets, and tables, respectively. The 
columns pertain to the hours of labor required for cutting the wood, assem- 
bling, and painting, respectively. The hourly wages of a carpenter to eut wood, 
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of a craftsperson to assemble a product, and of a decorator to paint is given, 
respectively, by the éléments of the matrix 



10.50 
14.00 . 
12.25 


Compute the product TW and détermine its significance. 

38 . Continuing with the data given in the previous problem, assume further that 
the number of items on order for lamp bases, cabinets, and tables, respectively, 
is given by the matrix O = [1000 100 200]. Compute the product OTW, 
and détermine its significance. 

39 . The results of a flu épidémie at a college campus are collected in the matrix 


F = 


0.20 

0.10 

0.70 


0.20 

0.30 

0.50 


0.15 

0.30 

0.55 


0.15 

0.40 

0.45 


The éléments dénoté percents converted to décimais. The columns pertain 
to freshmen, sophomores, juniors, and seniors, respectively, while the rows 
represent bedridden students, infected but ambulatory students, and well stu- 
dents, respectively. The male-female composition of each class is given by the 
matrix 


1050 

950 

1100 

1050 

360 

500 

860 

1000 


Compute the product FC, and détermine its significance. 


1.4 Spécial Matrices 

There are certain types of matrices that occur so frequently that it becomes advis- 
able to discuss them separately. One such type is the transpose. Given a matrix A, 
the transpose of A, denoted by A T and read A-transpose, is obtained by changing 
ail the rows of A into columns of A T while preserving the order; hence, the first 
row of A becomes the first column of A T , while the second row of A becomes 
the second column of A T , and the last row of A becomes the last column of A T . 
Thus if 


"1 

2 

3" 


"1 

4 

7" 

4 

5 

6 

, then A t = 

2 

5 

8 

7 

8 

9 


3 

6 

9 


A = 
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and if 

1 5" 

2 6 

3 7 • 

4 8 

Définition 1 If A, denoted by [a z/ ] is an n x p matrix, then the transpose of A, 
denoted by A T = [aj-] is a pxn matrix where aj- = aji. 

It can be shown that the transpose possesses the following properties: 

(1) (A t ) t = A, 

(2) (AA) t = àA t where X represents a scalar, 

(3) (A + B) t = A t + B t , 

(4) (A + B + C) T = A t + B t + C T , 

(5) (AB) t = B t A t , 

( 6 ) (ABC) t = C t B t A t 

Transposes of sums and products of more than three matrices are defined in the 
obvious manner. We caution the reader to be alert to the ordering of properties (5) 
and (6). In particular, one should be aware that the transpose of a product is not 
the product of the transposes but rather the commuted product of the transposes. 


A = 


'1 2 3 4' 

5 6 7 8 


then A t = 


Example 1 Find (AB) T and B T A T if 


Solution 




and 





AB = 

"-3 6 3" 
-17 4 

, (AB) t = 

"-3 -1" 
6 7 




3 4 


B t A t 



3' 

1 

0 


3 4 
0 1 




Note that (AB) T = B T A T but A T B T is not defined. ■ 


A zéro row in a matrix is a row containing only zéros, while a nonzero row is 
one that contains at least one nonzero element. A matrix is in row-reduced form 
if it satisfies four conditions: 

(RI) Ail zéro rows appear below nonzero rows when both types are présent in 
the matrix. 

(R2) The first nonzero element in any nonzero row is unity. 
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(R3) Ail éléments directly below ( that is, in the same column but in succeeding 
rows from) the first nonzero element of a nonzero row are zéro. 

(R4) The first nonzero element of any nonzero row appears in a later column 
(further to the right) than the first nonzero element in any preceding row. 


Such matrices are invaluable for solving sets of simultaneous linear équations and 
developing efficient algorithms for performing important matrix operations. We 
shall hâve much more to say on these matters in later chapters. Here we are simply 
interested in recognizing when a given matrix is or is not in row-reduced form. 


Example 2 Détermine which of the following matrices are in row-reduced form: 




1 

0 

0 

0 

1 

0 

0 


1 -2 4 
0-6 5 
0 0 0 
0 0 0 

2 3 4“ 

0 12 , 
1 0 5 


1 

7 

0 

0 



“1 

2 

3“ 

, B = 

0 

0 

0 


0 

0 

1 

r-i 

_ 

■2 

3 


D = | 0 0 1-3 

0 0 10 


Solution Matrix A is not in row-reduced form because the first nonzero element 
of the second row is not unity. This violâtes (R2). If 023 had been unity instead of 
—6, then the matrix would be in row-reduced form. Matrix B is not in row-reduced 
form because the second row is a zéro row and it appears before the third row 
which is a nonzero row. This violâtes (RI). If the second and third rows had been 
interchanged, then the matrix would be in row-reduced form. Matrix C is not in 
row-reduced form because the first nonzero element in row two appears in a later 
column, column 3, than the first nonzero element of row three. This violâtes (R4). 
If the second and third rows had been interchanged, then the matrix would be in 
row-reduced form. Matrix D is not in row-reduced form because the first nonzero 
element in row two appears in the third column, and everything below ^23 is not 
zéro. This violâtes (R3). Had the 3-3 element been zéro instead of unity, then the 
matrix would be in row-reduced form. ■ 


For the remainder of this section, we concern ourselves with square matrices; 
that is, matrices having the same number of rows as columns. A diagonal matrix is 
a square matrix ail of whose éléments are zéro except possibly those on the main 
diagonal. (Recall that the main diagonal consists of ail the diagonal éléments 
An, 022 , < 233 , and so on.) Thus, 

3 0 0“ 

0 3 0 
0 0 3 
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are both diagonal matrices of order 2x2 and 3x3 respectively. The zéro matrix 
is the spécial diagonal matrix having ail the éléments on the main diagonal equal 
to zéro. 

An identity matrix is a diagonal matrix worthy of spécial considération. Des- 
ignated by I, an identity is defined to be a diagonal matrix having ail diagonal 
éléments equal to one. Thus, 


1 0 
0 1 


and 


10 0 0 
0 10 0 
0 0 10 
0 0 0 1 


are the 2x2 and 4x4 identities respectively. The identity is perhaps the most 
important matrix of ail. If the identity is of the appropriate order so that the 
following multiplication can be carried out, then for any arbitrary matrix A, 

AI = A and IA = A. 


A symmeîric matrix is a matrix that is equal to its transpose while a skew symmetric 
matrix is a matrix that is equal to the négative of its transpose. Thus, a matrix A 
is symmetric if A = A T while it is skew symmetric if A = — A T . Examples of each 
are respectively 


"1 

2 

3" 


0 

2 

-3" 

2 

4 

5 

and 

-2 

0 

1 

3 

5 

6 


3 

-1 

0 


A matrix A = [ciij] is called lower triangular if aij — 0 for j > i (that is, if ail the 
éléments above the main diagonal are zéro) and upper triangular if aij — 0 for i > j 
(that is, if ail the éléments below the main diagonal are zéro). 

Examples of lower and upper triangular matrices are, respectively, 


5 

0 

0 

0 


~-l 

2 

4 

1" 

-1 

2 

0 

0 

and 

0 

1 

3 

-1 

0 

1 

3 

0 

0 

0 

2 

5 

2 

1 

4 

1 


0 

0 

0 

5 


Theorem 1 The product of two lower (upper) triangular matrices is also lower 
(upper) triangular. 

Proof. Let A and B both benxn lower triangular matrices. Set C = AB. We 
need to show that C is lower triangular, or equivalently, that c z/ = 0 when i < j. 
Now, 


n 


j~ 1 


n 


c ij — ^ ' a ik^kj — T ^ ' a ikhkj • 

k = 1 k = 1 k=j 
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We are given that a a — 0 when i < k, and bkj = 0 when k < y, because both A and 
B are lower triangular. Thus, 


j ~ 1 7-1 

^ ^ ttjk bkj — ^ ' ttjk (Q) — 0 
k= 1 k=l 

because k is always less than y. Furthermore, if we restrict i < y , then 


n 


n 


^ ' ttjkbkj — ^ ^ (Q)^Â7 — 0 
k=j k=j 


because k> j > i. Therefore, c/y = 0 when i <j. □ 

Finally, we define positive intégral powers of a matrix in the obvious manner: 
A 2 = AA, A 3 = AAA and, in general, if n is a positive integer, 


A n = AA ... A. 

-j 

V 

n times 

Thus, if 


"1 -2 

i . 0 

"1 -2~ 


"1 -2 


"-1 -8" 


, then A z — 




— 


1 3 


1 3 


1 3_ 


4^ 

-4 


It follows directly from Property 5 that 

(A 2 ) t = (AA) t = A t A t = (A t ) 2 . 


We can generalize this resuit to the following property for any intégral posi- 
tive power n : 

(7) (A") t = (A t )'\ 


Problems 1.4 


1. Verify that (A + B) T 

= A T + B t 

where 





"1 5 -1" 


6 

1 

3" 

A = 

2 1 3 

and B = 

2 

0 

-1 


0 7-8 


-1 

-7 

2 


2. Verify that (AB) T = B T A T , where 


t 

7 2 " 

i- 

1 

2 1 

and B = 

1 

0 _ 

- 


3 

t 


t 

2 t 


t + 1 
7 2 


0 


A = 
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3. Simplify the following expressions: 

(a) (AB t ) t , (b) A t + (A + B t ) t , 

(c) (A t (B + C T )) T , (d) ((AB) t + C) T , 

(e) ((A + A t )(A — A T )) T . 

4. Find X T X and XX T when 

X = 

5. Find X T X and XX T when X = [1 -2 3 -4], 

6. Find X T AX when 

r? 3i m 

A = ; À and X = . 

3 4 y 

7. Détermine which, if any, of the following matrices are in row-reduced form: 




8. Détermine which, if any, of the matrices in Problem 7 are upper triangular. 

9. Must a square matrix in row-reduced form necessarily be upper triangular? 
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11 . 

12 . 







18 . 

19 . 



21 . 

22 . 


Must an upper triangular matrix necessarily be in row-reduced form? 
Can a matrix be both upper and lower triangular simultaneously? 
Show that AB = BA, where 



~-l 

0 

0“ 


"5 

0 

0 " 

A = 

0 

3 

0 

and B = 

0 

3 

0 


0 

0 

1 


0 

0 

2 


Prove that if A and B are diagonal matrices of the same order, then AB = BA. 
Does a 2 x 2 diagonal matrix commute with every other 2x2 matrix? 
Compute the products AD and BD for the matrices 



1 

1 

1 


"0 

1 

2 


"2 

0 

0 " 

A = 

1 

1 

1 

, B = 

3 

4 

5 

, D = 

0 

3 

0 


1 

1 

1 


6 

7 

8 


0 

0 

-5 


What conclusions can you make about postmultiplying a square matrix by a 
diagonal matrix? 

Compute the products DA and DB for the matrices defined in Problem 15. 
What conclusions can you make about premultiplying a square matrix by a 
diagonal matrix? 

Prove that if a 2 x 2 matrix A commutes with every 2x2 diagonal matrix, then 
A must also be diagonal. Hint : Consider, in particular, the diagonal matrix 


Prove that if an n x n matrix A commutes with every n x n diagonal matrix, 
then A must also be diagonal. 

Compute D 2 and D 3 for the matrix D defined in Problem 15. 

Find A 3 if 



0 0 
2 0 . 
0 3 


Using the results of Problems 19 and 20 as a guide, what can be said about D 77 
if D is a diagonal matrix and n is a positive integer? 

Prove that if D = [dij] is a diagonal matrix, then D 2 = [dy\. 
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24. 












Calculate D 50 — 5D 35 + 41, where 



0 0 
0 1 
0 0 



A square matrix A is nilpotent if A” = 0 for some positive integer n. If n is 
the smallest positive integer for which A' 7 = 0 then A is nilpotent of index n. 
Show that 



is nilpotent of index 3. 
Show that 



0 

0 

0 

0 


1 0 
0 1 
0 0 
0 0 


0 

0 

1 

0 


is nilpotent. What is its index? 

Prove that if A is a square matrix, then B = (A + A T ) /2 is a symmetric matrix. 

Prove that if A is a square matrix, then C = (A — A T ) /2 is a skew symmetric 
matrix. 

Using the results of the preceding two problems, prove that any square matrix 
can be written as the sum of a symmetric matrix and a skew-symmetric matrix. 

Write the matrix A in Problem 1 as the sum of a symmetric matrix and skew- 
symmetric matrix. 

Write the matrix B in Problem 1 as the sum of a symmetric matrix and a 
skew-symmetric matrix. 

Prove that if A is any matrix, then AA T is symmetric. 

Prove that the diagonal éléments of a skew-symmetric matrix must be zéro. 

Prove that the transpose of an upper triangular matrix is lower triangular, and 
vice versa. 

If P = [pij] is a transition matrix for a Markov chain (see Problem 16 of Sec- 
tion 1.1), then it can be shown with elementary probability theory that the 
i — j element of P 2 dénotés the probability of an object moving from State i 
to stage j over two time periods. More generally, the i — j element of P' 7 for 
any positive integer n dénotés the probability of an object moving from State 
i to State j over n time periods. 
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(a) Calculate P 2 and P 3 for the two-state transition matrix 

0.1 0.9 
0.4 0.6 

(b) Détermine the probability of an object beginning in State 1 and ending in 
State 1 after two time periods. 

(c) Détermine the probability of an object beginning in State 1 and ending in 
State 2 after two time periods. 

(d) Détermine the probability of an object beginning in State 1 and ending in 
State 2 after three time periods. 

(e) Détermine the probability of an object beginning in State 2 and ending in 
State 2 after three time periods. 

35. Consider a two-state Markov chain. List the number of ways an object in State 
1 can end in State 1 after three time periods. 

36. Consider the Markov chain described in Problem 16 of Section 1.1. Détermine 
(a) the probability that a family living in the city will find themselves in the 
suburbs after two years, and (b) the probability that a family living in the 
suburbs will find themselves living in the city after two years. 

37. Consider the Markov chain described in Problem 17 of Section 1.1. Déter- 
mine (a) the probability that there will be a Republican mayor eight years 
after a Republican mayor serves, and (b) the probability that there will be a 
Republican mayor 12 years after a Republican mayor serves. 

38. Consider the Markov chain described in Problem 18 of Section 1.1. It is known 
that this year the apple harvest was poor. Détermine (a) the probability that 
next year’s harvest will be poor, and (b) the probability that the harvest in 
two years will be poor. 

39. Consider the Markov chain described in Problem 19 of Section 1.1. Détermine 
(a) the probability that a brand X customer will be a brand X customer after 
4 years, (b) after 6 years, and (c) the probability that a brand X customer will 
be a brand Y customer after 4 years. 

40. A graph consists of a set of nodes, which we shall designate by positive integers, 
and a set of arcs that connect various pairs of nodes. An adjacency matrix M 
associated with a particular graph is defined by 

m-ij = number of distinct arcs connecting node i to node j 

(a) Construct an adjacency matrix for the graph shown in Figure 1.1. 

(b) Calculate M 2 , and note that the i — j element of M 2 is the number of 
paths consisting of two arcs that connect node i to node j. 
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Figure 1.1 



Figure 1.2 


41. (a) Construct an adjacency matrix M for the graph shown in Figure 1.2. 

(b) Calculate M 2 , and use that matrix to détermine the number of paths 
consisting of two arcs that connect node 1 to node 5. 

(c) Calculate M 3 , and use that matrix to détermine the number of paths 
consisting of three arcs that connect node 2 to node 4. 



Figure 1.3 


42. Figure 1.3 depicts a road network linking various cities. A traveler in city 1 
needs to drive to city 7 and would like to do so by passing through the least 
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number of intermediate cities. Construct an adjacency matrix for this road 
network. Consider powers of this matrix to solve the traveler’s problem. 

1.5 Submatrices and Partitioning 

Given any matrix A, a submatrix of A is a matrix obtained from A by the removal 
of any number of rows or columns. Thus, if 

1 2 3 4] 

5 6 7 8 TIC) 12] a 'x ai 

9 10 11 12 ’ B- |_14 16j and C_[2 3 4] ’ 

13 14 15 lôj 

then B and C are both submatrices of A. Here B was obtained by removing from 
A the first and second rows together with the first and third columns, while C was 
obtained by removing from A the second, third, and fourth rows together with the 
first column. By removing no rows and no columns from A, it follows that A is a 
submatrix of itself. 

A matrix is said to be partitioned if it is divided into submatrices by horizontal 
and vertical lines between the rows and columns. By varying the choices of where 
to put the horizontal and vertical lines, one can partition a matrix in many different 
ways. Thus, 

' 1 2 3 41 T 1 2 3 4" 

J 8 . ~5 6 7 8 

9 10 11 12 9 10 11 12 

_13 14 15 lôj |_13 14 15 16_ 

are examples of two different partitions of the matrix A given in (11). 

If partitioning is carried out in a particularly judicious manner, it can be a great 
help in matrix multiplication. Consider the case where the two matrices A and B 
are to be multiplied together. If we partition both A and B into four submatrices, 
respectively, so that 

rc di 

A = and B = 

Le f J 

where C through K represent submatrices, then the product AB may be obtained 
by simply carrying out the multiplication as if the submatrices were themselves 
éléments. Thus, 

AB = 

providing the partitioning was such that the indicated multiplications are defined. 

It is not unusual to need products of matrices having thousands of rows and 
thousands of columns. Problem 42 of Section 1.4 dealt with a road network con- 
necting seven cities. A similar network for a State with connections between ail 
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cities in the State would hâve a very large adjacency matrix associated with it, and 
its square is then the product of two such matrices. If we expand the network 
to include the entire United States, the associated matrix is huge, with one row 
and one column for each city and town in the country. Thus, it is not difficult to 
visualize large matrices that are too big to be stored in the internai memory of any 
modem day computer. And yet the product of such matrices must be computed. 

The solution procedure is partitioning. Large matrices are stored in external 
memory on peripheral devices, such as disks, and then partitioned. Appropriate 
submatrices are fetched from the peripheral devices as needed, computed, and the 
results again stored on the peripheral devices. An example is the product given in 
(12). If A and B are too large for the internai memory of a particular computer, 
but C through K are not, then the partitioned product can be computed. First, 
C and G are fetched from external memory and multiplied; the product is then 
stored in external memory. Next, D and J are fetched and multiplied. Then, the 
product CG is fetched and added to the product DJ. The resuit, which is the first 
partition of AB, is then stored in external memory, and the process continues. 


Example 1 Find AB if 



"3 

1 

2" 


1 

3 

2" 

A = 

1 

4 

-1 

and B = 

-1 

0 

1 


3 

1 

2 


0 

1 

1 


Solution We first partition A and B in the following manner 



"3 1 

2" 


" 1 3 

2' 

A = 

1 4 

-1 

and B = 

-1 0 

1 


_3 1 

2_ 


t-H 

O 
1 

1 


then, 


"3 1 
1 4 

1 3" 
-1 0 

+ 

2" 

-1 

[0 1 ] 

"3 1" 
1 4 

~2 

1 

+ 

2" 

-1 

[ i ]' 

[3 1 ] 

1 3" 
— ! 0 

+ [ 2][0 1 ] 

[3 1 ] 

"2" 

1 

+ P ] [ 1 ] 


2 9 
[-3 3 

+ 

0 2 
0 -lj 

7 

_ 6 J + 

2 

-1 

[2 9 

" 2 11 
-3 2 

4- i 

1 ON IT) 

[0 

2] 

2 1 
-3 

[7] H 

1 9" 

2 5 

H [2] 

. ■ 

2 11 

9j 

2 11 9j 
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Example 2 Find AB if 


"3 

1 

0 " 






2 

0 

0 


" 2 

1 

0 

0 

0 

0 

0 

3 

and B = 

-1 

1 

0 

0 

0 

0 

0 

1 


0 

1 

0 

0 

1 

0 

0 

0 







Solution From the indicated partitions, we find that 


AB = 



"3 1" 
2 0 

2 1" 
-1 1 

+ 

"0" 

0 

[0 1] 


"3 r 
2 0 

"0 0 0" 
0 0 0 

+ 

"0" 

0 

[° 0 ■] 


i i 

o o 

0 o 

1 1 

2 1“ 
-1 1 

+ 

"3" 

4 

[0 1] 


1 1 

o o 

o o 
1 1 

"0 0 0" 
0 0 0 

+ 

"3" 

1 

[° 0 ■] 

[0 0] 

2 r 

_-i i 

+ [0] [0 1] 

[0 0] 

"0 0 0" 
0 0 0 

+ [0] [0 0 1] 


AB = 



"5 4" 
4 2 

+ 

1 1 

o o 

0 o 

1 1 


i i 

o o 

o o 
1 1 

+ 

"0 3" 
0 1 

[0 0] + [0 0] 


"0 

0 

0" 

+ 

"0 

0 

0" 

““ 

0 

0 

0 

0 

0 

0 


"0 

0 

0" 

+ 

"0 

0 

3" 


0 

0 

0 

0 

0 

1 


0 

0 

°] + [° 

0 

0] 



5 

4 

0 

0 

0 " 


"5 

4 

0 

0 

0 

4 

2 

0 

0 

0 


4 

2 

0 

0 

0 

0 

3 

0 

0 

3 

— 

0 

3 

0 

0 

3 

0 

1 

0 

0 

1 


0 

1 

0 

0 

1 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 


Note that we partitioned in order to make maximum of the zéro submatrices of 
both A and B. ■ 


A matrix A that can be partitioned into the form 





0 


0 



is called block diagonal. Such matrices are particularly easy to multiply because 
in partitioned form they act as diagonal matrices. 
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Problems 1.5 


1. Which of the following are submatrices of the given A and why? 



2 3 
5 6 
8 9 


(a) 


1 3' 
7 9 


(b) [1] 


(c) 


'1 2 ' 
8 9 


2. Détermine ail possible submatrices of 



b 

d 


(d) 


4 6 
7 9 


3. Given the matrices A and B (as shown), find AB using the partitionings 
indicated: 



"1 

-1 

2" 


"5 

2 

0 

2' 

A = 

3 

0 

4 

, B = 

1 

-1 

3 

1 


_0 

1 

2_ 


_0 

1 

1 

4 


4. Partition the given matrices A and B and, using the results, find AB. 


"4 

1 

0 

1 

O 


3 

2 

0 

0" 

2 

2 

0 

0 

, B = 

-1 

1 

0 

0 

0 

0 

1 

0 

0 

0 

2 

1 

r o~ 

0 

1 

2 


0 

0 

1 

-1 


5. Compute A 2 for the matrix A given in Problem 4 by partitioning A into block 
diagonal form. 

6. Compute B 2 for the matrix B given in Problem 4 by partitioning B into block 
diagonal form. 

7. Use partitioning to compute A 2 and A 3 for 

"1 0 0 0 0 0 " 

0 2 0 0 0 0 

0 0 0 1 0 0 

A_ 000010 ' 

0 0 0 0 0 1 

0 0 0 0 0 0 


What is A” for any positive intégral power of n > 3? 
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8. Use partitioning to compute A 2 and A 3 for 



0 

-1 

0 

0 

0 

0 

0 


-10 0 
0 0 0 
0 2-2 
0-1 3 

0 1 -2 
0 0 0 
0 0 0 



0 0 

0 0 

0 0 

0 0 . 

0 0 

-1 0 

0 -1 


What is A n for any positive intégral power of ni 


1 .6 Vectors 

Définition 1 A vector is a 7 x n or n x 1 matrix. 

Al x n matrix is called a row vector while annxl matrix is a column vector. The 
éléments are called the components of the vector while the number of components 
in the vector, in this case n , is its dimension. Thus, 

"1" 

2 

3 

is an example of a 3-dimensional column vector, while 

[t 2 1 —t 0] 

is an example of a 4-dimensional row vector. 

The reader who is already familiar with vectors will notice that we hâve not 
defined vectors as directed line segments. We hâve done this intentionally, first 
because in more than three dimensions this géométrie interprétation loses its sig- 
nificance, and second, because in the general mathematical framework, vectors 
are not directed line segments. However, the idea of representing a finite dimen- 
sional vector by its components and hence as a matrix is one that is acceptable 
to the scientist, engineer, and mathematician. Also, as a bonus, since a vector is 
nothing more than a spécial matrix, we hâve already defined scalar multiplication, 
vector addition, and vector equality. 

A vector y (vectors will be designated by boldface lowercase letters) has asso- 
ciated with it a nonnegative number called its magnitude or length designated 

by ||y||. 


Définition 2 If y = [y, y 2 ... y„] then ||y|| = vW 2 + (w) 2 + • • • + (y,,) 2 - 
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Example 1 Find ||y|| if y = [ 1 2 3 4 ]. 


Solution y = V (l) 2 + (2) 2 + (3) 2 + (4) 2 = V3Ô. 


If z is a column vector, ||z|| is defined in a completely analogous manner. 


Example 2 Find ||z|| if 


z = 


- 1 ' 

2 

-3 


Solution ||z|| = V(-l) 2 + (2) 2 + (-3) 2 = VÏ4. ■ 


A vector is called a wraY vector if its magnitude is equal to one. A nonzero 
vector is said to be normalized if it is divided by its magnitude. Thus, a normalized 
vector is also a unit vector. 


Example 3 Normalize the vector [1 0—3 2 —1]. 

Solution The magnitude of this vector is 

Vd) 2 + (O) 2 + (-3)2 + (2) 2 + (-1) 2 = V15. 


Hence, the normalized vector is 



-3 2 -1 " 

Vl5 VÏ5 VÏ5. 


In passing, we note that when a general vector is written y = [yiy2 • • • yn] one of 
the subscripts of each element of the matrix is deleted. This is done solely for the 
sake of convenience. Since a row vector has only one row (a column vector has 
only one column), it is redundant and unnecessary to exhibit the row subscript 
(the column subscript). 


Problems 1.6 

1. Find p if 5x — 2y = b, where 


"1 


"2 


1 " 

3 

0 

> y = 

P 

1 

, and b = 

13 

-2 


x = 
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2. Find x if 3x + 2y = b, where 


"3 


2" 

1 

and b = 

-1 

4 

6 

0 


1 


3. Find y if 2x — 5y = —b, where 

x = [2 —1 3] and b = [1 0 —1] . 


4 . Using the vectors defined in Problem 2, calculate, if possible, 

(a) yb, (b) yb T , 

(c) y T b, (d) b T y. 

5. Using the vectors defined in Problem 3, calculate, if possible, 

(a) x + 2b, (b) xb T , 

(c) x T b, (d) b T b. 


6. Détermine which of the following are unit vectors: 

(a) [1 1], (b) [1/2 1/2], (c)[l/V2 -1/V2] 


"0 


"1/2 


"1/V3" 

1 

(e) 

1/3 

(f) 

1/V3 

0 


_!/6_ 


_1/V3_ 



1 

2 


T 

1 

„ x 1 

T 

5 

1 

• <h) 6 

3 

1 


1 




0 1 



7 . Find ||y|| if 

(a) y = [l -l], (b)y=[3 4], 

( c ) y = [- 1 -i i], (d)y = fi I 2]’ 

(e)y = [2 1 -1 3], (f) y = [0 -1 5 3 



8. Find x if 



r il 


ni 


1 

h- > ‘ 

1 

(a)x = 

-1 

, (b) x = 

2 

, (c) X = 

1 

1 
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11 


1 

1 


~r 

-1 

, (e)x = 

2 

, (f)x = 

0 

1 

3 

i 

-1 


i 

i 


— i 
O 
1 


9. Find ||y|| if 

(a) y = [2 1 -1 3], (b) y = [0 -1 5 3 2]. 

10 . Prove that a normalized vector must be a unit vector. 


11. Show that the matrix équation 


"11-2 




"-3“ 

2 5 3 


J 

— 

11 

-1 3 1 


z_ 


5 


is équivalent to the vector équation 



1 


"1 


"-2 


~-3“ 


2 

+ y 

5 

4- z 

3 

— 

11 


-1 


3 


1 


5 


12 . Convert the following System of équations into a vector équation: 

2x + 3 y = 10, 

4x + 5y = 11. 

13 . Convert the following System of équations into a vector équation: 

3x + 4y + 5z + 6w — 1, 
y — 2z + 8u; = 0, 

—v + y 4- 2z — w = 0. 

14 . Using the définition of matrix multiplication, show that the yth column of 
(AB) = A x (yth column of B). 

15 . Verify the resuit of Problem 14 by showing that the first column of the product 
AB with 


is 


A = 


1 2 
4 5 



and B 
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while the second column of the product is 




16. A distribution row vector d for an A-state Markov chain (see Problem 16 of 
Section 1.1 and Problem 34 of Section 1.4) is an A-dimensional row vector 
having as its components, one for each State, the probabilities that an object in 
the System is in each of the respective States. Détermine a distribution vector 
for a three-state Markov chain if 50% of the objects are in State 1, 30% are in 
State 2, and 20% are in State 3. 

17. Let d (A) dénoté the distribution vector for a Markov chain after k time periods. 
Thus, d (0) represents the initial distribution. It follows that 

— (j(0)p£ — p(* — l)p 

where P is the transition matrix and is its kth power. 

Consider the Markov chain described in Problem 16 of Section 1.1. 

(a) Explain the physical significance of saying d (0) = [0.6 0.4]. 

(b) Find the distribution vectors d (1) and d (2) . 

18. Consider the Markov chain described in Problem 19 of Section 1.1. 

(a) Explain the physical significance of saying d (0) = [0.4 0.5 0.1]. 

(b) Find the distribution vectors d (1) and d (2) . 

19. Consider the Markov chain described in Problem 17 of Section 1.1. 

(a) Détermine an initial distribution vector if the town currently has a Démo- 
cratie mayor, and (b) show that the components of d (1) are the probabilities 
that the next mayor will be a Republican and a Democrat, respectively. 

20. Consider the Markov chain described in Problem 18 of Section 1.1. 

(a) Détermine an initial distribution vector if this year’s crop is known to 
be poor. (b) Calculate d (2) and use it to détermine the probability that the 
harvest will be good in two years. 


1 .7 The Geometry of Vectors 

Vector arithmetic can be described geometrically for two- and three-dimensional 
vectors. For simplicity, we consider two dimensions here; the extension to three- 
dimensional vectors is straightforward. For convenience, we restrict our examples 
to row vectors, but note that ail constructions are equally valid for column vectors. 

A two dimensional vector y — [a b ] is identified with the point ( a , b) on the 
plane, measured from the origin a units along the horizontal axis and then b 
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units parallel to the vertical axis. We can then draw an arrow beginning at the 
origin and ending at the point {a, b). This arrow or directed line segment, as 
shown in Figure 1.4, represents the vector geometrically. It follows immediately 
from Pythagoras’s theorem and Définition 2 of Section 1.6 that the length of the 
directed line segment is the magnitude of the vector. The angle associated with a 
vector, denoted by 0 in Figure 1.4, is the angle from the positive horizontal axis to 
the directed line segment measured in the counterclockwise direction. 


Example 1 Graph the vectors v = [2 4] and u = [— 1 1] and détermine the 

magnitude and angle of each. 

Solution The vectors are drawn in Figure 1.5. Using Pythagoras’s theorem and 
elementary trigonometry, we hâve, for v, 

|| v || = V(2) 2 + (4) 2 = 4.47, tan<9=^=2, and (9 = 63.4°. 

For u, similar computations yield 

|| u || =V / (-l ) 2 + (l ) 2 = 1.14, tan(9= 2_ = _i, an d 0 = 135°. ■ 

To construct the sum of two vectors u + v geometrically, graph u normally, 
translate v so that its initial point coincides with the terminal point of u, hein g 
careful to preserve both the magnitude and direction of v, and then draw an arrow 
from the origin to the terminal point of v after translation. This arrow geometrically 
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Figure 1.5 



Figure 1.6 


represents the sum u + v. The process is depicted in Figure 1.6 for the two vectors 
defined in Example 1. 

To construct the différence of two vectors u — v geometrically, graph both u 
and v normally and construct an arrow from the terminal point of v to the terminal 
point of u. This arrow geometrically represents the différence u — v. The process 
is depicted in Figure 1.7 for the two vectors defined in Example 1. To measure the 
magnitude and direction of u — v, translate it so that its initial point is at the origin, 
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Figure 1.7 


being careful to preserve both its magnitude and direction , and then measure the 
translated vector. 

Both geometrical sums and différences involve translations of vectors. This 
suggests that a vector is not altered by translating it to another position in the 
plane providing both its magnitude and direction are preserved. 

Many physical phenomena such as velocity and force are completely described 
by their magnitudes and directions. For example, a velocity of 60 miles per 
hour in the northwest direction is a complété description of that velocity, and 
it is independent of where that velocity occurs. This independence is the ratio- 
nale behind translating vectors geometrically. Geometrically, vectors having 
the same magnitude and direction are called équivalent , and they are regarded 
as being equal even though they may be located at different positions in the 
plane. 

A scalar multiplication ku is defined geometrically to be a vector having length 
\\k\\ times the length of u with direction equal to u when h is positive, and opposite 
to u when k is négative. Effectively, ku is an élongation of u by a factor of ||&|| 
when \\k\\ is greater than unity, or a contraction of u by a factor of ||Æ|| when ||/:|| 
is less than unity, folio wed by no rotation when k is positive, or a rotation of 180 
degrees when k is négative. 


Example 2 Find — 2u and \ v geometrically for the vectors defined in Example 1. 

Solution To construct — 2u, we double the length of u and then rotate the result- 
ing vector by 180°. To construct \y we halve the length of v and effect no rotation. 
These constructions are illustrated in Figure 1.8. ■ 
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Figure 1.8 


Problems 1.7 


In Problems 1 through 16, geometrically construct the indicated vector operations 
for 


u = [3 -1], v = [—2 5], w = [—4 -4], 


x = 


'3' 

5 


and y = 


0 ' 

-2 


1. u + v. 

5.x -y. 
9. u — w. 

13. ju. 


2. u + w. 

6. y - x. 
10 . 2x. 

14. — ju. 


3. v + w. 

7. u — y. 
11. 3x. 

15. ^v. 


4. x + y. 

8. w — u. 

12. — 2x. 

16. —\w. 


17. Détermine the angle of u. 


18. Détermine the angle of y. 


19. Détermine the angle of w. 20. Détermine the angle of x. 

21. Détermine the angle of y. 

22. For arbitrary two-dimensional row vectors construct on the same graph u + v 
and v + u. 

(a) Show that u + v = v + u. 

(b) Show that the sum is a diagonal of a parallelogram having u and v as two 
of its sides. 
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Simultaneous Linear Equations 


2.1 Linear Systems 

Systems of simultaneous équations appear frequently in engineering and scientific 
problems. Because of their importance and because they lend themselves to 
matrix analysis, we devote this entire chapter to their solutions. 

We are interested in Systems of the form 


011*1 4- 012*2 H h a\ n x n = b \, 

021*1 + 022*2 H h 02rc*« = b 2, 

(i) 


0m 1*1 i 0m2*2 i * * * i 0mn*n — 

We assume that the coefficients aij (i = 1,2 , ,m; j — 1 , 2, . . . , n) and the quan- 

tifies b[ (i = 1, 2, , m) are ail known scalars. The quantifies *i, * 2 , . . . , x n 
represent unknowns. 

Définition 1 A solution to (1) is a set of n scalars x\,X 2 , ...,*« that when 
substituted into ( 1 ) satisfies the given équations (that is, the equalities are valid). 

System ( 1 ) is a generalization of Systems considered earlier in that m can differ 
from n. If m > n, the System has more équations than unknowns. If m < n, the 
System has more unknowns than équations. If m — n, the System has as many 
unknowns as équations. In any case, the methods of Section 1.3 may be used to 
convert ( 1 ) into the matrix form 


Ax = b, 
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where 


A = 

an a \2 • • • a\ n 

a 21 a 22 ‘ ‘ ‘ a 2n 

• • • 

• • • 

, X = 

XI 

x 2 

• 

• 

, b = 

b\ 

b 2 


_a m \ ®m 2 ' ‘ * tt,nn_ 


_X n _ 


h 

_ u m _ 


Thus, if m ^ n, A will be rectangular and the dimensions of x and b will be different. 


Example 1 Convert the following System to matrix form: 


Solution 


x + 2y — z + w = 4, 
x + 3y + 2z 4- 4tu = 9. 



1 2 
1 3 


-1 1 
2 4 ’ 


x 




Example 2 Convert the following System to matrix form: 

x — 2 y — —9, 

4 x + y = 9, 

2x + y = 7, 

x — y = —1. 

Solution 


"1 

-2 




"-9" 

4 

1 


X 

, b = 

9 

2 

1 

, x = 

A. 

7 

1 

-1 




-1 


A System of équations given by (1) or (2) can possess no solutions, exactly 
one solution, or more than one solution (note that by a solution to (2) we mean a 
vector x which satisfies the matrix equality (2)). Examples of such Systems are 


x + y = l, 
x -h y = 2, 



x + y = l, 
x - y = 0, 


( 4 ) 
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x + y = 0, 
2x + 2 y — 0. 
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( 5 ) 


Equation (3) has no solutions, (4) admits only the solution x = y = while (5) 
has solutions x = — y for any value of y. 

Définition 2 A System of simultaneous linear équations is consistent if it 
possesses at least one solution. If no solution exists, the System is inconsistent. 

Equation (3) is an example of an inconsistent System, while (4) and (5) 
represent examples of consistent Systems. 

Définition 3 A System given by (2) is homogeneous if b = 0 (the zéro vec- 
tor). If b 0 (at least one component of b differs from zéro) the System is 
nonhomogeneous . 

Equation (5) is an example of a homogeneous System. 


Problems 2.1 


In Problems 1 and 2, détermine whether or not the proposed values of x, y, and z 
are solutions of the given Systems. 


1. x+ y + 2z = 2, (a)x = l,y = — 3, z = 2. 

x — y — 2z = 0, (b) x = 1 , y = — 1 , z = 1. 

x + 2y + 2z — 1. 


2. x + 2y + 3z = 6, 
x — 3y H - 2z — 0, 
3x — 4y + 7z = 6. 


(a) x = 1, y = 1, z= 1. 

(b) x = 2, y = 2, z = 0. 

(c) x = 14, y = 2, z = -4. 


3. Find a value for k such that x = 1, y = 2, and z = k is a solution of the System 


2x + 2y + 4z = 1, 
5x + y -h 2z = 5, 
x — 3y — 2z — —3. 


4. Find a value for k such that x = 2 and y — k is a solution of the System 


3x + 5y = 11, 
2x — 7y = —3. 
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5. Find a value for k such that x — 2 k,y = —k, and z = 0 is a solution of the 
System 

x + 2 y + z = 0, 

—2x — 4y + 2z = 0, 

3x — 6y — 4z = 1. 

6. Find a value for k such that x — 2 k, y = — k , and z = 0 is a solution of the 
System 

x + 2y + 2z — 0, 

2x — 4y + 2z = 0, 

— 3x — 6y — 4z = 0. 

7. Find a value for k such that x = 2k, y = — k , and z = 0 is a solution of the 
System 

x + 2y + 2z = 0, 

2x + 4y + 2z = 0, 

— 3x — 6y — 4z = 1. 

8. Put the System of équations given in Problem 4 into the matrix form Ax = b. 

9. Put the System of équations given in Problem 1 into the matrix form Ax = b. 

10 . Put the System of équations given in Problem 2 into the matrix form Ax = b. 

11. Put the System of équations given in Problem 6 into the matrix form Ax = b. 

12 . A manufacturer receives daily shipments of 70,000 springs and 45,000 pounds 
of stuffing for producing regular and support mattresses. Regular mattresses 
r require 50 springs and 30 pounds of stuffing; support mattresses s require 
60 springs and 40 pounds of stuffing. The manufacturer wants to know how 
many mattresses of each type should be produced daily to utilize ail available 
inventory. Show that this problem is équivalent to solving two équations in 
the two unknowns r and 5. 

13 . A manufacturer produces desks and bookcases. Desks d require 5 hours of 
cutting time and 10 hours of assembling time. Bookcases b require 15 minutes 
of cutting time and one hour of assembling time. Each day, the manufacturer 
has available 200 hours for cutting and 500 hours for assembling. The manu- 
facturer wants to know how many desks and bookcases should be scheduled 
for completion each day to utilize ail available workpower. Show that this 
problem is équivalent to solving two équations in the two unknowns d and b. 

14 . A mining company has a contract to supply 70,000 tons of low-grade ore, 
181,000 tons of medium-grade ore, and 41,000 tons of high-grade ore to a 
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supplier. The company has three mines which it can work. Mine A produces 
8000 tons of low-grade ore, 5000 tons of medium-grade ore, and 1000 tons of 
high-grade ore during each day of operation. Mine B produces 3000 tons of 
low-grade ore, 12,000 tons of medium-grade ore, and 3000 tons of high-grade 
ore for each day it is in operation. The figures for mine C are 1000, 10,000, 
and 2000, respectively. Show that the problem of determining how many days 
each mine must be operated to meet contractual demands without surplus 
is équivalent to solving a set of three équations in A, S, and C, where the 
unknowns dénoté the number of days each mine will be in operation. 

15. A pet store has determined that each rabbit in its care should receive 80 units 
of protein, 200 units of carbohydrates, and 50 units of fat daily. The store 
carries four different types of feed that are appropriate for rabbits with the 
following compositions: 


Feed 

Protein 

units/oz 

Carbohydrates 

units/oz 

Fat 

units/oz 

A 

5 

20 

3 

B 

4 

30 

3 

C 

8 

15 

10 

D 

12 

5 

7 


The store wants to détermine a blend of these four feeds that will meet the 
daily requirements of the rabbits. Show that this problem is équivalent to 
solving three équations in the four unknowns A, B , C, and Z), where each 
unknown dénotés the number of ounces of that feed in the blend. 

16. A small company computes its end-of-the-year bonus b as 5% of the net profit 
after city and State taxes hâve been paid. The city tax c is 2% of taxable income, 
while the State tax s is 3% of taxable income with crédit allowed for the city 
tax as a pretax déduction. This year, taxable income was $400,000. Show that 
b , c, and s are related by three simultaneous équations. 

17. A gasoline producer has $800,000 in fixed annual costs and incurs an additional 
variable cost of $30 per barrel B of gasoline. The total cost C is the sum of 
the fixed and variable costs. The net sales S is computed on a Wholesale price 
of $40 per barrel. (a) Show that C, B , and S are related by two simultaneous 
équations, (b) Show that the problem of determining how many barrels must 
be produced to break even, that is, for net sales to equal cost, is équivalent to 
solving a System of three équations. 

18. (Leontief Closed Models) A closed économie model involves a society in 
which ail the goods and services produced by members of the society are con- 
sumed by those members. No goods and services are imported from without 
and none are exported. Such a System involves N members, each of whom pro- 
duces goods or services and charges for their use. The problem is to détermine 
the prices each member should charge for his or her labor so that everyone 


48 


Chapter 2 Simultaneous Linear Equations 


breaks even after one year. For simplicity, it is assumed that each member 
produces one unit per year. 

Consider a simple closed System consisting of a farmer, a carpenter, and a 
weaver. The farmer produces one unit of food each year, the carpenter pro- 
duces one unit of finished wood products each year, and the weaver produces 
one unit of clothing each year. Let p\ dénoté the farmer’s annual income 
(that is, the price she charges for her unit of food), let p 2 dénoté the car- 
penter’s annual income (that is, the price he charges for his unit of finished 
wood products), and let p 3 dénoté the weaver’s annual income. Assume on 
an annual basis that the farmer and the carpenter consume 40% each of the 
available food, while the weaver eats the remaining 20%. Assume that the 
carpenter uses 25% of the wood products he makes, while the farmer uses 
30% and the weaver uses 45%. Assume further that the farmer uses 50% of 
the weaver’s clothing while the carpenter uses 35% and the weaver consumes 
the remaining 15%. Show that a break-even équation for the farmer is 


0.40/71 + 0.30/72 + 0.50/73 = p\, 


while the break-even équation for the carpenter is 


0.40/71 + 0.25/72 + 0.35/73 = p2. 


What is the break-even équation for the weaver? Rewrite ail three équations 
as a homogeneous System. 

19 . Paul, Jim, and Mary décidé to help each other build houses. Paul will spend 
half his time on his own house and a quarter of his time on each of the houses of 
Jim and Mary. Jim will spend one third of his time on each of the three houses 
under construction. Mary will spend one sixth of her time on Paul’s house, one 
third on Jim’s house, and one half of her time on her own house. For tax pur- 
poses each must place a price on his or her labor, but they want to do so in a way 
that each will break even. Show that the process of determining break-even 
wages is a Leontief closed model comprised of three homogeneous équations. 

20 . Four third world countries each grow a different fruit for export and each 
uses the income from that fruit to pay for imports of the fruits from the other 
countries. Country A exports 20% of its fruit to country B, 30% to country 
C, 35% to country D, and uses the rest of its fruit for internai consumption. 
Country B exports 10% of its fruit to country A, 15% to country C, 35% to 
country D, and retains the rest for its own citizens. Country C does not export 
to country A; it divides its crop equally between countries B and D and its 
own people. Country D does not consume its own fruit. Ail of its fruit is for 
export with 15% going to country A, 40% to country B, and 45% to country C. 
Show that the problem of determining priées on the annual harvests of fruit 
so that each country breaks even is équivalent to solving four homogeneous 
équations in four unknowns. 



2.1 


Linear Systems 


49 


21. (Leontief Input-Output Models) Consider an economy consisting of N sec- 
tors, with each producing goods or services unique to that sector. Let x/ dénoté 
the amount produced by the ith sector, measured in dollars. Thus x/ represents 
the dollar value of the supply of product i available in the economy. Assume 
that every sector in the economy has a demand for a proportion (which may 
be zéro) of the output of every other sector. Thus, each sector j has a demand, 
measured in dollars, for the item produced in sector i. Let a X j dénoté the pro- 
portion of item f s revenues that must be committed to the purchase of items 
from sector i in order for sector j to produce its goods or services. Assume 
also that there is an external demand, denoted by di and measured in dollars, 
for each item produced in the economy. 

The problem is to détermine how much of each item should be pro- 
duced to meet external demand without creating a surplus of any item. Show 
that for a two sector economy, the solution to this problem is given by the 
supply/demand équations 

supply demand 

X\ — <211X1 + <212X2 + 

X 2 = <221X1 + <222*2 + Û?2- 

Show that this System is équivalent to the matrix équations 

x = Ax + d and (I — A)x = d. 

In this formulation, A is called the consumption matrix and d the demand 
vector. 

22. Détermine A and d in the previous problem if sector 1 must expend half of its 
revenues purchasing goods from its own sector and one third of its revenues 
purchasing goods from the other sector, while sector 2 must expend one quar- 
ter of its revenues purchasing items from sector 1 and requires nothing from 
itself. In addition, the demand for items from these two sectors are $20,000 
and $30,000, respectively. 

23. A small town has three primary industries, coal mining (sector 1), transporta- 
tion (sector 2), and electricity (sector 3). Production of one dollar of coal 
requires the purchase of 10 cents of electricity and 20 cents of transportation. 
Production of one dollar of transportation requires the purchase of 2 cents of 
coal and 35 cents of electricity. Production of one unit of electricity requires 
the purchase of 10 cents of electricity, 50 cents of coal, and 30 cents of trans- 
portation. The town has external contracts for $50,000 of coal, $80,000 of 
transportation, and $30,000 units of electricity. Show that the problem of 
determining how much coal, electricity, and transportation is required to sup- 
ply the external demand without a surplus is équivalent to solving a Leontief 
input-output model. What are A and d? 

24. An economy consists of four sectors: energy, tourism, transportation, and 
construction. Each dollar of income from energy requires the expenditure 
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of 20 cents on energy costs, 10 cents on transportation, and 30 cents on con- 
struction. Each dollar of income gotten by the tourism sector requires the 
expenditure of 20 cents on tourism (primarily in the form of complimentary 
facilities for favored customers), 15 cents on energy, 5 cents on transporta- 
tion, and 30 cents on construction. Each dollar of income from transportation 
requires the expenditure of 40 cents on energy and 10 cents on construction; 
while each dollar of income from construction requires the expenditure of 5 
cents on construction, 25 cents on energy, and 10 cents on transportation. The 
only external demand is for tourism, and this amounts to $5 million dollars a 
year. Show that the problem of determining how much energy, tourism, trans- 
portation, and construction is required to supply the external demand without 
a surplus is équivalent to solving a Leontief input-output model. What are A 
and d? 

25. A constraint is often imposed on each column of the consumption matrix of 
a Leontief input-output model, that the sum of the éléments in each column 
be less than unity. Show that this guarantees that each sector in the economy 
is profitable. 


2.2 Solutions by Substitution 

Most readers hâve probably encountered simultaneous équations in high school 
algebra. At that time, matrices were not available; hence other methods were 
developed to solve these Systems, in particular, the method of substitution. We 
review this method in this section. In the next section, we develop its matrix 
équivalent, which is slightly more efficient and, more importantly, better suited 
for computer implémentations. 

Consider the System given by (1): 


011*1 + 012*2 H h a\ n x n = b\, 

021*1 + < 322*2 H h ^ 2 n x n = & 2 > 


0m 1*1 T 0m2*2 H - * * * H - 0 mn x n — b 


m 


The method of substitution is the following: take the first équation and solve for 
x\ in terms of X 2 , X 3 , . . . , x n and then substitute this value of x\ into ail the other 
équations, thus eliminating it from those équations. (If jq does not appear in the 
first équation, rearrange the équations so that it does. For example, one might 
hâve to interchange the order of the first and second équations.) This new set of 
équations is called the first derived set. Working with the first derived set, solve 
the second équation for X 2 in terms of jq, *4 , . . . , x n and then substitute this value 
of X 2 into the third, fourth, etc. équations, thus eliminating it. This new set is the 
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second derived set. This process is kept up until the following set of équations is 
obtained: 


X\ — C 12*2 “bci 3**3 + C14X4 + • • • + C\ n X n + Ji, 

•*2 = 

X3 = 

x m — 

where the a fs and the df s are some combination of the original <z//s and /?/’s. 
System (6) can be quickly solved by back substitution. 


Example 1 Use the method of substitution to solve the System 

r ~\~ 2 s -\- t — 3, 

2r H- 3s — t — —6, 

3r — 2s — 4t — —2. 

Solution By solving the first équation for r and then substituting it into the 
second and third équations, we obtain the first derived set 

r = 3 — 2s — t, 

—s — 3t — —12, 

-8s — 7t= —11. 

By solving the second équation for s and then substituting it into the third équation, 
we obtain the second derived set 


c 23 x 3 + c 24 x 4 + • * * + c 2 n x n + d 2 , 

C34V4 + • • • H- C?,n x n + ^3> 


( 6 ) 


Cm,m-\-l x tn -\- 1 H - ‘ ‘ ‘ H - ^ mn x n “b d m , 


r — 3 — 2s — t, 
s — 12 — 3?, 

17f = 85. 

By solving for t in the third équation and then substituting it into the remaining 
équations (of which there are none), we obtain the third derived set 

r = 3 — 2s — t, 
s — 12 — 3/\ 
t = 5. 


Thus, the solution is t — 5, s = — 3, r = 4. ■ 
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Example 2 Use the method of substitution to solve the System 

x + y + 3z = — 1, 

2x — 2y — z — 1, 

5x + y -h 8z = —2. 

Solution The first derived set is 


x — —1 — y — 3z, 

—4 y - Iz = 3, 

— 4y - lz = 3. 

The second derived set is 

v = — 1 — y — 3z, 

3 7 

J =“4“4 Z ’ 

0 = 0 . 

Since the third équation can not be solved for z, this is as far as we can go. Thus, since 
we can not obtain a unique value for z, the first and second équations will not yield a 
unique value for v and y. Caution: The third équation does not imply that z = 0. On 
the contrary, this équation says nothing at ail about z, consequently z is completely 
arbitrary. The second équation gives y in terms of z. Substituting this value into 
the first équation, we obtain v in terms of z. The solution therefore is v = — ^ — |z 
and y = — | — ^z, z is arbitrary. Thus there are infinitely many solutions to the 
above System. However, once z is chosen, v and y are determined. If z is chosen 
to be —1, then v = y = 1, while if z is chosen to be 3, then v = —4, y = —6. The 
solutions can be expressed in the vector form 


r*i 


i 

l 

-p^i i — ^ 

l 

J 


r in 
4 


i 

1 

i 

i 

i 

— 

_3 _Z 7 
4 4 Z 

Z 

— 

3 

4 

0_ 

+ z 

i 

1 

-p^i — i 

i 


Example 3 Use the method of substitution to solve 


a + 2b — 3c + d — 1, 
2 a T 6 b T 4c T 2 d — 8. 
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Solution The first derived set is 


a — 1 — 2b + 3c — d, 
2b + 10c = 6. 


The second derived set is 


a — 1 — 2b + 3c — d 


b — 3 — 5c 


Again, since there are no more équations, this is as far as we can go, and since 
there are no defining équations for c and d , these two unknowns must be arbitrary. 
Solving for a and/? in termsofc and d, weobtainthe solutions = — 5 + 13c — d, b — 
3 — 5c; c and d are arbitrary. The solutions can be expressed in the vector form 


a 


— 5 T 13c — d 


~-5" 


" 13" 



b 


3-5 c 


3 


-5 

T d 

0 

c 

— 

c 

— 

0 

+ 

1 

0 

d 


d 


0 


0 


1 


Note that while c and d are arbitrary, once they are given a particular value, a and 
b are automatically determined. For example, if c is chosen as —1 and d as 4, a 
solution is a — —22, Z? = 8, c — — 1, J = 4, while if c is chosen as 0 and d as —3, a 
solution is a — —2, b — 3, c = 0, J = —3. ■ 


Example 4 Use the method of substitution to solve the following System: 

x + 3y = 4, 

2x — y = 1, 

3x + 2y = 5, 

5x + 15y = 20. 

Solution The first derived set is 

x = 4 — 3y, 

-7 y = 

-7 y = - 7 , 

0 = 0 . 
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The second derived set is 


x — 4 — 3 y, 
y -h 

o = o, 

0 = 0 . 

Thus, the solution is y = l,x = 1, or in vector form 


X 


T 

y. 


1 


Problems 2.2 

Use the method of substitution to solve the following Systems: 

1. x + 2y — 2z = — 1, 2. x-\- y— z— 0, 

3x 4- 2y 4~ 4z = 0. 


2x 4- y 4- z = 5, 

— x + y — z — —2. 

3. x 4~ 3y = 4, 

2x — y — 1, 

—2x — 6y = —8, 

4x — 9 y — —5, 

—6x -h 3 y = —3. 

5. 2/— m 4~ n — p — 1, 

/ + 2m — n + 2 p = — 1, 
/ — 3m + 2n — 3/7 = 2. 

7. x + 2y — z = 5, 

2x — y + 2z = 1, 

2x4-2 y — z—1 , 
x + 2y+ z=3. 


4. 4r — 3,s + 2f = 1 , 
r 4- s — 2t— 4, 
5r — 2s — t = 5. 


6 . 2x + y — z = 0, 
x + 2y+ z = 0, 

3x — y + 2z = 0. 

8. x + 2y+ z — 2 m = 1, 
2x + 2y — z— w— 3, 
2x — 2y + 2z + 3 w = 3, 
3x4- y — 2z — 3w = 1. 


2.3 Gaussian Elimination 

Although the method of substitution is straightforward, it is not the most efficient 
way to solve simultaneous équations, and it does not lend itself well to electronic 
computing. Computers hâve difficulty symbolically manipulating the unknowns 
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in algebraic équations. A striking feature of the method of substitution, however, 
is that the unknowns remain unaltered throughout the process: x remains x, y 
remains y, z remains z- Only the coefficients of the unknowns and the numbers on 
the right side of the équations change from one derived set to the next. Thus, we 
can save a good deal of writing, and develop a useful représentation for computer 
Processing, if we direct our attention to just the numbers themselves. 

Définition 1 Given the System Ax = b, the augmented matrix , designated by 
A b , is a matrix obtained from A by adding to it one extra column, namely b. 


Thus, if 


then 


while if 


then 



2 3 
5 6 


and b = 


'7' 

8 



12 3 7 
4 5 6 8’ 



1 

2 

3" 


- 1 " 

A = 

4 

5 

6 

and b = 

-2 


7 

8 

9 


-3 



12 3-1 
456-2. 
7 8 9 -3 


In particular, the System 


x + y — 2z — —3, 
2x + 5y + 3z = 11, 
— x + 3y + z = 5. 


has the matrix représentation 


"11-2 


x 


~-3“ 

2 5 3 


J 


11 

-1 3 1 


z_ 


5 
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with an augmented matrix of 



11-2-3 
25 3 11 . 

-13 15 


Example 1 Write the set of équations in x, y , and z associated with the 
augmented matrix 


Solution 



-2 13 8 

045 — 3 ' 


— 2x+ y -h 3z = 8, 


4y + 5z = -3. 


A second striking feature to the method of substitution is that every derived set 
is different from the System that preceded it. The method continues creating new 
derived sets until it has one that is particularly easy to solve by back-substitution. 
Of course, there is no purpose in solving any derived set, regardless how easy it 

is, unless we are assured beforehand that it has the same solution as the original 
System. Three elementary operations that alter équations but do not change their 
solutions are: 

(i) Interchange the positions of any two équations. 

(ii) Multiply an équation by a nonzero scalar. 

(iii) Add to one équation a scalar times another équation. 

If we restate these operations in words appropriate to an augmented matrix, 
we obtain the elementary row operations: 

(El) Interchange any two rows in a matrix. 

(E2) Multiply any row of a matrix by a nonzero scalar. 

(E3) Add to one row of a matrix a scalar times another row of that same matrix. 

Gaussian élimination is a matrix method for solving simultaneous linear équa- 
tions. The augmented matrix for the System is created, and then it is transformed 
into a row-reduced matrix (see Section 1.4) using elementary row operations. This 
is most often accomplished by using operation (E3) with each diagonal element 
in a matrix to create zéros in ail columns directly below it, beginning with the 
first column and moving successively through the matrix, column by column. The 
System of équations associated with a row-reduced matrix can be solved easily by 
back-substitution, if we solve each équation for the first unknown that appears in 

it. This is the unknown associated with the first nonzero element in each nonzero 
row of the final augmented matrix. 
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Example 


Solution 


Then, 


2 Use Gaussian élimination to solve 

x + 3 y — 4, 

2x — y — 1, 

3x + 2 y — 5, 

5x + 15y = 20. 

The augmented matrix for this System is 


1 

2 

3 

5 


3 

-1 

2 

15 


4' 

1 

5 

20 


1 



2 

-1 

1 


3 

2 

5 


5 

15 

20 

"1 

3 

4" 

r 

0 - 

-7 

-7 


3 

2 

5 

• 

|_5 15 

20 


"1 

3 

4" 

r 

0 - 

-7 

-7 


0 - 

-7 

-7 

• 

|_5 15 

20 


"1 

3 

4" 

r 

0 - 

-7 

-7 


0 - 

-7 

-7 

• 

0 

0 

0 


"1 

3 

4" 

£ 

0 

1 

1 


0 - 

-7 

-7 


0 

0 

0 

l 


second row (—2) times 
the first row 


third row (—3) times 
the first row 


fourth row (—5) times 
the first row 


[ by multiplying the 

. -1 


7 


"1 

3 

i 


0 

1 

1 


0 

0 

0 

• 

• 

1 0 

0 

0 



second row (7) times 
the first row 
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The System of équations associated with this last augmented matrix in 
row-reduced form is 

x + 3y = 4, 

y = i, 
o = o, 

0 = 0 . 

Solving the second équation for y and then the first équation for x, we obtain x — 1 
and y = 1, which is also the solution to the original set of équations. Compare this 
solution with Example 4 of the previous section. ■ 

The notation (— >) should be read “is transformed into”; an equality sign is not 
correct because the transformed matrix is not equal to the original one. 

Example 3 Use Gaussian élimination to solve 

r + 2s + t — 3, 

2r H- 3s — t — —6, 

3r — 2s — 4t = —2. 

Solution The augmented matrix for this System is 

12 13“ 

2 3 -1 -6 . 

3 -2 -4 -2 

Then, 

1213 1213 by adding to the 

2 3 —1 —6 —> 0 —1 —3 —12 second row (—2) times 

3 —2 —4 —2 3 —2 —4 —2 the first row 

— —J L_ _J 

12 13 by adding to the 

— > 0 —1 —3 —12 third row (— 3) times 

0 —8 —7 —11 the first row 

_ _j 

by multiplying the 
second row by (— 1) 
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"12 13 

-> 0 1 3 12 

0 0 17 85 


12 13 
-> 0 1 3 12 . 

0 0 15 

The System of équations associated 
reduced form is 


59 

by adding to the 
third row (8) times 
the second row 

, 

I by multiplying the 
third row by 

with this last augmented matrix in row- 


r -h 2s -h t — 3, 
s + 3t = 12, 
t — 5. 


Solving the third équation for t , then the second équation for and, lastly, the first 
équation for r, we obtain r — 4, s = —3, and t — 5, which is also the solution to the 
original set of équations. Compare this solution with Example 1 of the previous 
section. ■ 


Whenever one element in a matrix is used to cancel another element to zéro by 
elementary row operation (E3), the first element is called the pivot. In Example 
3, we first used the element in the 1-1 position to cancel the element in the 
2-1 position, and then to cancel the element in the 3-1 position. In both of 
these operations, the unity element in the 1-1 position was the pivot. Later, we 
used the unity element in the 2-2 position to cancel the element —8 in the 3-2 
position; here, the 2-2 element was the pivot. 

While transforming a matrix into row-reduced form, it is advisable to adhéré 
to three basic principles: 

• Completely transform one column to the required form before considering 
another column. 

• Work on columns in order, from left to right. 

• Never use an operation if it will change a zéro in a previously transformed 
column. 

As a conséquence of this last principle, one never involves the ith row of a 
matrix in an elementary row operation after the ith column has been transformed 
into its required form. That is, once the first column has the proper form, no pivot 
element should ever again corne from the first row; once the second column has 
the proper form, no pivot element should ever again corne from the second row; 
and so on. 

When an element we want to use as a pivot is itself zéro, we interchange rows 
using operation (El). 
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Example 4 Use Gaussian élimination to solve 

2c + 3cf = 4, 
ci + 3c + cf = 2, 
cf H - /? -J- 2c — 0. 

Solution The augmented matrix is 

"0 0 2 3 4" 
10 3 12. 
112 0 0 


Normally, we would use the element in the 1-1 position to cancel to zéro the 
two éléments directly below it, but we cannot because it is zéro. To proceed with 
the réduction process, we must interchange the first row with either of the other 
two rows. The choice is arbitrary. 


0 

0 

2 

3 

4 ^ 

1 


1 

0 

3 1 

2 


1 

0 

3 

1 

2 

-* 

0 

0 

2 3 

4 


1 

1 

2 

0 

0 


1 

1 

2 0 

0 







"1 

0 

3 

1 

2' 





-* 

0 

0 

2 

3 

4 






0 

1 

-1 

-1 

-2 


by interchanging the 
first row with the 
second row 

, 

by adding to the 
third row (—1) times 
the first row 


Next, we would like to use the element in the 2-2 position to cancel to zéro the 
element in the 3-2 position, but we cannot because that prospective pivot is zéro. 
We use elementary row operation (El) once again. The transformation yields 


10 3 12 

0 1 - 1-1 -2 
0 0 2 3 4 

10 3 12 

0 1 - 1-1 -2 
0 0 1 1.5 2 


by interchanging the 
second row with the 
third row 

[ by multiplying the 
I third row by (0.5) 


The System of équations associated with this last augmented matrix in row- 
reduced form is 


ci T 3c T d — 2, 

b — c — d — —2, 

c T 1.5 d — 2. 
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We use the third équation to solve for c, the second équation to solve for b , and 
the first équation to solve for a , because these are the unknowns associated with 
the first nonzero element of each nonzero row in the final augmented matrix. We 
hâve no defining équation for d , so this unknown remains arbitrary. The solution 
is, a — — 4 -h 3.5 d, b — —0.5 d, c — 2 — 1.5 d, and d arbitrary, or in vector form 


a 


—4 + 3.5 d 


-4 


7" 

b 


-0.5 d 


0 

d 

-1 

c 

— 

2-1.5 d 

— 

2 

+ 2 

-3 

d 


d 


0 


2 


This is also the solution to the original set of équations. ■ 

The derived set of équations associated with a row-reduced, augmented matrix 
may contain an absurd équation, such as 0 = 1. In such cases, we conclude that the 
derived set is inconsistent, because no values of the unknowns can simultaneously 
satisfy ail the équations. In particular, it is impossible to choose values of the 
unknowns that will make the absurd équation true. Since the derived set has the 
same solutions as the original set, it follows that the original set of équations is 
also inconsistent. 


Example 5 Use Gaussian élimination to solve 

2x + 4y + 3z = 8, 
3x — 4y — 4z = 3, 
5 a" — z = 12. 


Solution The augmented matrix for this System is 

"2 4 3 8" 

3-4-4 3 . 

5 0 -1 12 


Then, 


"2 

4 

3 

8" 


"1 

2 

1.5 4" 


3 

-4 

-4 

3 

-* 

3 

-4 

-4 3 


5 

0 

-1 

12 


_5 

0 

-1 12 






"1 

2 

1.5 

4 




-> 

0 

-10 

-8.5 - 

-9 





5 

0 

-1 

12 


I by multiplying the 
first row by 

by adding to the 
second row (—3) times 
the first row 
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1 2 1.5 4 

0 -10 -8.5 -9 

0 -10 -8.5 -8 

1 2 1.5 4 

0 1 0.85 0.9 

0 -10 -8.5 -8 

1 2 1.5 4" 

0 1 0.85 0.9 . 

0 0 0 1 


( by adding to the 
third row (—5) times 
the first row 

by multiplying the 
second row by 

by adding to the 
third row (10) times 
the second row 


The System of équations associated with this last augmented matrix in row- 
reduced form is 


x + 2y + 1.5z = 4, 
y + 0.85z = 0.9, 

0 = 1 . 

Since no values of x, y, and z can make this last équation true, this System, as well 
as the original one, has no solution. ■ 

Finally, we note that most matrices can be transformed into a variety of row- 
reduced forms. If a row-reduced matrix has two nonzero rows, then a different 
row-reduced matrix is easily constructed by adding to the first row any nonzero 
constant times the second row. The équations associated with both augmented 
matrices, however, will hâve identical solutions. 


Problems 2.3 

5, construct augmented matrices for the given Systems of 

2. x 4-2 y — z — —1, 

2x — 3y + 2z = 4. 

4 . 2r + 4s = 2, 

3r + 2s 4- t — 8, 

5r — 3s + It — 15. 

5. 2 r + 3 s — 4 1 — 12, 

3r — 2s — — 1, 

8r — s — 4î — 10. 


In Problems 1 through 
équations: 

1. x -\-2 y — —3, 

3x4- y— 1. 

3. a -\- 2b— 5, 

—3 a + b — 13, 

4a 4- 3b — 0. 
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In Problems 6 through 11, write the set of équations associated with the given 
augmented matrix and the specified variables. 


6. A b 

7. A b 

8. A b 

9. A b 

10. A b 

11. A b 


'1 

0 

0 

'1 

0 

0 

0 


'1 2 5' 

0 1 8 

"1 -2 3 10' 

0 1-5-3 

0 0 14 


variables: v and y. 


variables: x, y , and z. 


1 -3 12 


40' 


0 1 -6 -200 

0 0 1 25 


variables: r, s, and t. 


'13 0 - 8 ' 
0 14 2 


0 0 0 


0 


variables: x, y , and z. 


•7 

1 

0 

1 

1 

0 

0 


2 0 " 
-1 0 
0 0 


variables: a, b , and c. 


0 

-2 

1 

0 


1' 

2 

-3 

1 


variables: u, v, and w. 


12. Solve the System of équations defined in Problem 6. 

13. Solve the System of équations defined in Problem 7. 

14. Solve the System of équations defined in Problem 8. 

15. Solve the System of équations defined in Problem 9. 

16. Solve the System of équations defined in Problem 10. 

17. Solve the System of équations defined in Problem 11. 

In Problems 18 through 24, use elementary row operations to transform the given 
matrices into row-reduced form: 




5 

8 


19. 

"4 

24 

20" 

20. 

"0 

-1 

6" 

• 9 

2 

11 

-8 

• mm w • 

2 

7 

-5 


1 

2 

3 

4 ^ 
1 


1 

O 

1 

-2 

4" 

-1 

-1 

2 

3 

22. 

1 

3 

2 

1 

-2 

3 

0 

i 

o 


-2 

3 

1 

2 


21 . 
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2 

3 

4 

6 

0 

10 

24. 

-5 

-8 

15 

1 

3 

40 


3 

3 

5 

4 

4 

20 


25. Solve Problem 1. 26. Solve Problem 2. 

27. Solve Problem 3. 28. Solve Problem 4. 

29. Solve Problem 5. 

30. Use Gaussian élimination to solve Problem 1 of Section 2.2. 

31. Use Gaussian élimination to solve Problem 2 of Section 2.2. 

32. Use Gaussian élimination to solve Problem 3 of Section 2.2. 

33. Use Gaussian élimination to solve Problem 4 of Section 2.2. 

34. Use Gaussian élimination to solve Problem 5 of Section 2.2. 

35. Détermine a production schedule that satisfies the requirements of the 
manufacturer described in Problem 12 of Section 2.1. 

36. Détermine a production schedule that satisfies the requirements of the 
manufacturer described in Problem 13 of Section 2.1. 

37. Détermine a production schedule that satisfies the requirements of the 
manufacturer described in Problem 14 of Section 2.1. 

38. Détermine feed blends that satisfy the nutritional requirements of the pet 
store described in Problem 15 of Section 2.1. 


39. Détermine the bonus for the company described in Problem 16 of 
Section 2.1. 


40. Détermine the number of barrels of gasoline that the producer described in 
Problem 17 of Section 2.1 must manufacture to break even. 


41. Détermine the annual incomes of each sector of the Leontief closed model 
described in Problem 18 of Section 2.1. 

42. Détermine the wages of each person in the Leontief closed model described 
in Problem 19 of Section 2.1. 


43. Détermine the total sales revenue for each country of the Leontief closed 
model described in Problem 20 of Section 2.1. 

44. Détermine the production quotas for each sector of the economy described 
in Problem 22 of Section 2.1. 


45. An elementary matrix is a square matrix E having the property that the product 
EA is the resuit of applying a single elementary row operation on the matrix 
A. Form a matrix H from the 4x4 identity matrix I by interchanging any two 
rows of I, and then compute the product HA for any 4x4 matrix A of your 
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choosing. Is H an elementary matrix? How would one construct elementary 
matrices corresponding to operation (El)? 

46. Form a matrix G from the 4x4 identity matrix I by multiplying any one row 
of I by the number 5, and then compute the product GA for any 4x4 matrix 
A of your choosing. Is G an elementary matrix? How would one construct 
elementary matrices corresponding to operation (E2)? 

47. Form a matrix F from the 4x4 identity matrix I by adding to one row of I 
five times another row of I. Use any two rows of your choosing. Compute 
the product FA for any 4x4 matrix A of your choosing. Is F an elemen- 
tary matrix? How would one construct elementary matrices corresponding to 
operation (E3)? 

48. A solution procedure uniquely suited to matrix équations of the form x = 
Ax + d is itération. A trial solution x (0) is proposed, and then progressively 
better estimâtes x (1) , x (2) , x (3) , ... for the solution are obtained iteratively 
from the formula 

x (i+1) = Ax (0 + d. 

The itérations terminate when two successive estimâtes differ by less than a 
prespecified acceptable tolérance. 

If the System cornes from a Leontief input-output model, then a reasonable 
initialization is x (0) = 2d. Apply this method to the System defined by Problem 
22 of Section 2.1. Stop after two itérations. 

49. Use the itération method described in the previous problem to solve the 
System defined in Problem 23 of Section 2.1. In particular, find the first two 
itérations by hand calculations, and then use a computer to complété the 
itération process. 

50. Use the itération method described in Problem 48 to solve the System defined 
in Problem 24 of Section 2.1. In particular, find the first two itérations 
by hand calculations, and then use a computer to complété the itération 
process. 


2.4 Pivoting Strategies 

Gaussian élimination is often programmed for computer implémentation. Since ail 
computers round or truncate numbers to a finite number of digits (e.g., the fraction 
1/3 might be stored as 0.33333, but never as the infinité décimal 0.333333 . . .) 
roundoff error can be significant. A number of strategies hâve been developed to 
minimize the effects of such errors. 

The most popular strategy is partial pivoting , which requires that a pivot élé- 
ment always be larger in absolute value than any element below it in the same 
column. This is accomplished by interchanging rows whenever necessary. 
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Example 1 Use partial pivoting with Gaussian élimination to solve the System 

x + 2y + 4z = 18, 

2x + 12y — 2 z= 9, 

5x + 26y + 5z = 14. 

Solution The augmented matrix for this System is 

"12 4 18" 

2 12 -2 9 . 

5 26 5 14 


Normally, the unity element in the 1-1 position would be the pivot. With partial 
pivoting, we compare this prospective pivot to ail éléments directly below it in the 
same column, and if any is larger in absolute value, as is the case here with the 
element 5 in the 3-1 position, we interchange rows to bring the largest element 
into the pivot position. 


1 

2 

5 


Then, 


2 

12 

26 


4 

18 


"5 26 


5 1 

-2 

9 

-* 

2 12 


Z 

5 

14 


1 2 

4 1 


"1 

5.2 

1 

2.8" 


-* 

2 

12 

-2 

9 



1 

2 

4 

18 


"1 

5.2 

1 2.8' 


• 

0 

1.6 

— 

1 3.4 


* 

1 

2 


1 18 


, 


by interchanging the 
first and third rows 


by multiplying the 


second row (—2) times 
the first row 


1 

5.2 

1 

2.8 

0 

1.6 

-4 

3.4 

0 

-3.2 

3 

15.2 


by adding to the 
third row (—1) times 
the first row 


The next pivot would normally be the element 1.6 in the 2-2 position. Before 
accepting it, however, we compare it to ail éléments directly below it in the same 
column. The largest element in absolute value is the element —3.2 in the 3-2 
position. Therefore, we interchange rows to bring this larger element into the 
pivot position. 

Note. We do not consider the element 5.2 in the 1-2 position, even though it 
is the largest element in its column. Comparisons are only made between a 
prospective pivot and ail éléments directly below it. Recall one of the three basic 
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principles of row-reduction: ne ver involve the first row of matrix in a row operation 
after the first column has been transformée! into its required form. 


1 

5.2 

1 2.8 


0 

-3.2 

3 15.2 


0 

1.6 

-4 3.4 


1 

5.2 

1 2.8 ' 

0 

1 

-0.9375 -4.75 

0 

1.6 

-4 3.4 

1 

5.2 

1 2.8 ' 

0 

1 

-0.9375 -4.75 

0 

0 

-2.5 

h j 

1 

5.2 

1 

2.8 ' 

0 

1 

-0.9375 - 

4.75 

0 

0 

1 

4.4 


by interchanging the 
second and third rows 

by multiplying the 
second row by ^ 

by adding to the 
third row (—1.6) times 
the second row 

. 

by multiplying the 
third row by 


The new derived set of équations is 

x + 5.2y + z — 2.8, 
y - 0.9375z = -4.75, 
z = -4.4, 


which has as its solution x = 53.35, y = —8.875, and z = —4.4. ■ 

Scaled pivoting involves ratios. A prospective pivot is divided by the largest 
element in absolute value in its row, ignoring the last column. The resuit is com- 
pared to the ratios formed by dividing every element directly below the pivot by 
the largest element in absolute value in its respective row, again ignoring the last 
column. Of these, the element that yields the largest ratio in absolute value is 
designated as the pivot, and if that element is not already in the pivot position, 
then row interchanges are performed to move it there. 


Example 2 Use scaled pivoting with Gaussian élimination to solve the System 
given in Example 1. 

Solution The augmented matrix for this System is 

"12 4 18" 

2 12 -2 9 . 

5 26 5 14 


Normally, we would use the element in the 1-1 position as the pivot. With 
scaled pivoting, however, we first compare ratios between éléments in the first 
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column to the largest éléments in absolute value in each row, ignoring the last 
column. The ratios are 

- = 0.25, — = 0.1667, and — = 0.1923. 

4 12 26 

The largest ratio in absolute value corresponds to the unity element in the 1-1 
position, so that element remains the pivot. Transforming the first column into 
reduced form, we obtain 


12 4 18 

0 8 -10 -27 . 

0 16 -15 -76 

Normally, the next pivot would be the element in the 2-2 position. Instead, we 
consider the ratios 


8 16 , 

— = 0.8 and — = 1, 

10 16 

which are obtained by dividing the pivot element and every element directly below 
it by the largest element in absolute value appearing in their respective rows, ignor- 
ing éléments in the last column. The largest ratio in absolute value corresponds to 
the element 16 appearing in the 3-2 position. We move it into the pivot position 
by interchanging the second and third rows. The new matrix is 

"12 4 18" 

0 16 -15 -76 . 

0 8 -10 -27 

Completing the row-reduction transformation, we get 

"12 4 18 

0 1 -0.9375 -4.75 . 

0 0 1 -4.4 

The System of équations associated with this matrix is 

v + 2 y + 4 z— 18, 

y - 0.9375z = -4.75, 
z = —4.4. 

The solution is, as before, x — 53.35, y = —8.875, and z = —4.4. ■ 
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Complété pivoting compares prospective pivots with ail éléments in the largest 
submatrix for which the prospective pivot is in the upper left position, ignoring 
the last column. If any element in this submatrix is larger in absolute value than 
the prospective pivot, both row and column interchanges are made to move this 
larger element into the pivot position. Because column interchanges rearrange the 
order of the unknowns, a book keeping method must be implemented to record 
ail rearrangements. This is done by adding a new row, designated as row 0, to the 
matrix. The entries in the new row are initially the positive integers in ascending 
order, to dénoté that column 1 is associated with variable 1, column 2 with variable 
2, and so on. This new top row is only affected by column interchanges; none of 
the elementary row operations is applied to it. 


Example 3 Use complété pivoting with Gaussian élimination to solve the 
System given in Example 1. 

Solution The augmented matrix for this System is 


"1 

2 

3 


1 

2 

4 

18 

2 

12 

-2 

9 

_5 

26 

5 

!4_ 


Normally, we would use the element in the 1-1 position of the coefficient matrix 
A as the pivot. With complété pivoting, however, we first compare this prospective 
pivot to ail éléments in the submatrix shaded below. In this case, the element 26 is 
the largest, so we interchange rows and columns to bring it into the pivot position. 


1 

2 

3 



"1 

2 

3 


1 

2 

4 

18 


5 

26 

5 

14 

2 

12 

-2 

9 


2 

12 

-2 

9 

5 

26 

5 

14_ 


_1 

2 

4 

18_ 





" 2 

1 

3 

— i 





26 

5 

5 

14 





12 

2 

-2 

9 





2 

1 

4 

18 


f by interchanging the 
I first and third rows 


f by interchanging the 
I first and second columns 


Applying Gaussian élimination to the first column, we obtain 


"2 

1 

3 


1 

0.1923 

0.1923 

0.5385 

0 

-0.3077 

-4.3077 

2.5385 

_0 

0.6154 

3.6154 

16.9231_ 
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Normally, the next pivot would be —0.3077. Instead, we compare this number 
in absolute value to ail the numbers in the submatrix shaded above. The largest 
such element in absolute value is —4.3077, which we move into the pivot position 
by interchanging the second and third column. The resuit is 


"2 

3 

1 


1 

0.1923 

0.1923 

0.5385 

0 

-4.3077 

-0.3077 

2.5385 

_0 

3.6154 

0.6154 

16.9231_ 


Continuing with Gaussian élimination, we obtain the row-reduced matrix 


"2 

3 

1 


1 

0.1923 

0.1923 

0.5385 

0 

1 

0.0714 

-0.5893 

_0 

0 

1 

53.35 


The System associated with this matrix is 

y + 0.1923z + 0.1923* = 0.5385, 
z + 0.0714* = -0.5893, 

* = 53.35. 

Its solution is, * = 53.35, y = —8.8749, and z = —4.3985, which is within round-off 
error of the answers gotten previously. ■ 


Complété pivoting generally identifies a better pivot than scaled pivoting 
which, in turn, identifies a better pivot than partial pivoting. Nonetheless, par- 
tial pivoting is most often the strategy of choice. Pivoting strategies are used to 
avoid roundoff error. We do not need the best pivot; we only need to avoid bad 
pivots. 


Problems 2.4 

In Problems 1 through 6, détermine the first pivot under (a) partial pivoting, 
(b) scaled pivoting, and (c) complété pivoting for given augmented matrices. 
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3 35" 

8 15 ’ 



-2 -5" 

3 85 ' 



8 15" 
-4 11 ' 



-2 

8 

-3 

100 

4. 

4 

5 

4 

75 


-3 

-1 

2 

250 



1 

5 

9 


2 

6 

10 


3 4 

7 8 . 

11 12 



”0 

2 

3 

4 

0 

6. 

1 

0.4 

0.8 

0.1 

90 


4 

10 

1 

8 

40 


7 . Solve Problem 3 of Section 2.3 using Gaussian élimination with each of the 
three pivoting strategies. 

8. Solve Problem 4 of Section 2.3 using Gaussian élimination with each of the 
three pivoting strategies. 

9 . Solve Problem 5 of Section 2.3 using Gaussian élimination with each of the 
three pivoting strategies. 

10 . Computers internally store numbers in formats similar to the scientific 
notation 0, -E-, representing the number 0. -multiplied by the power of 10 
signified by the digits following E. Therefore, 0.1234E06 is 123,400 while 
0.9935E02 is 99.35. The number of digits between the décimal point and E 
is finite and fixed; it is the number of significant figures. Arithmetic opera- 
tions in computers are performed in registers, which hâve twice the number 
of significant figures as storage locations. 

Consider the System 


0 . 0000 Lr + y = 1 . 00001 , 
x + y = 2 . 

Show that when Gaussian élimination is implemented on this System by a computer 
limited to four significant figures, the resuit is x — 0 and y = 1 , which is incorrect. 
Show further that the difficulty is resolved when partial pivoting is employed. 


2.5 Linear Independence 

We momentarily digress from our discussion of simultaneous équations to develop 
the concepts of linearly independent vectors and rank of a matrix, both of which 
will prove indispensable to us in the ensuing sections. 

Définition 1 A vector Vi is a linear combination of the vectors V2, V3, . . . , \ n 
if there exist scalars ^ 2 ,^ 3 , . . . , d n such that 


Vl = J 2 V 2 + CI 3 Y 3 H + d n \n. 
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Example 1 Show that [1 2 3] is a linear combination of [2 4 0] and 

[0 0 1]. 

Solution [1 2 3] = \[2 4 0] + 3[0 0 1], ■ 


Ref erring to Example 1 , we could say that the row vector [1 2 3] dépends lin- 

early on the other two vectors or, more generally, that the set of vectors {[1 2 3], 

[2 4 0], [0 0 1]} linear ly dépendent. AnotYitr vi oi expressing thisdepen- 
dence would be to say that there exist constants ci, C 2 , C 3 not ail zéro such that 
ci [1 2 3] +C 2 [2 4 0] +C 3 [0 0 1] = [0 0 0]. Such a set would be 

ci = — 1, C 2 = c 3 — 3. Note that the set ci = C 2 = C 3 = 0 is also a suitable set. 
The important fact about dépendent sets, however, is that there exists a set of 
constants, not ail equal to zéro , that satisfies the equality. 

Now consider the set given by Vi = [1 0 0 ]V 2=[0 1 0 ]V 3=[0 0 1]. 
It is easy to verify that no vector in this set is a linear combination of the other two. 
Thus, each vector is linearly independent of the other two or, more generally, the 
set of vectors is linearly independent. Another way of expressing this independence 
would be to say the only scalars that satisfy the équation ci [1 0 01 4 - C 2 [0 1 0 ] 

+ c 3 [0 0 1 ] = [0 0 0 ] are ci = c 2 = c 3 = 0 . 

Définition 2 A set of vectors {Vi, V 2 , . . . , V«}, of the same dimension, is lin- 
early dépendent if there exist scalars ci, C 2 , . . . , c„, not ail zéro, such that 


ciVi 4 - C2V2 + C 3 V 3 + • • • + c n \ n — 0 



The vectors are linearly independent if the only set of scalars that satisfies (7) is 
the set ci = C 2 = • • • = c n — 0 . 

Therefore, to test whether or not a given set of vectors is linearly independent, 
first form the vector équation (7) and ask “What values for the c’s satisfy this 
équation?” Clearly ci = C 2 = • • • = c n — 0 is a suitable set. If this is the only set of 
values that satisfies (7) then the vectors are linearly independent. If there exists a 
set of values that is not ail zéro, then the vectors are linearly dépendent. 

Note that it is not necessary for ail the c’s to be different from zéro for a 
set of vectors to be linearly dépendent. Consider the vectors Vi = [1, 2], Y 2 = 
[ 1 , 4 ], V3 = [2, 4 ]. ci = 2, C2 = 0, C3 = — 1 is a set of scalars, not ail zéro, such that 
c i Vi + C2 V 2 + C3V3 = 0. Thus, this set is linearly dépendent. 


Example 2 Is the set {[1, 2], [ 3, 4]} linearly independent? 
Solution The vector équation is 


cifl 2] + c 2 [3 4] = [0 0]. 
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This équation can be rewritten as 

[ci 2 ci] + [ 3 c 2 4 c 2 ] = [0 0 ] 


or as 


[ci + 3 c 2 2c! + 4 c 2 ] = [0 0 ]. 

Equating components, we see that this vector équation is équivalent to the System 


a + 3c 2 = o, 
2 ci + 4 c 2 = 0 . 


Using Gaussian élimination, we find that the only solution to this System is ci = 
c 2 = 0, hence the original set of vectors is linearly independent. ■ 

Although we hâve worked exclusively with row vectors, the above définitions 
are equally applicable to column vectors. 


Example 3 Is the set 



linearly independent? 

Solution Consider the vector équation 



2" 


"3 


8” 


"0" 

Cl 

6 

+ c 2 

1 

+ C 3 

16 

— 

0 


-2 


2 


-3 


0 



This équation can be rewritten as 


2ci 


3 c 2 


8C3 


" 0 " 

6ci 

+ 

Q 

+ 

16 c 3 

— 

0 

_-2ci_ 


_2c2_ 


_- 3 c 3 _ 


0 


2c\ + 3c2 + 8c 3 


" 0 " 

6c\ + C2 + 1ÔC3 

— 

0 

_-2c\ + 2c 2 - 3c3_ 


0 


or as 
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By equating components, we see that this vector équation is équivalent to the 
System 

2c\ + 3c2 + 8C3 = 0, 

6ci + C2 + I6C3 = 0, 

— 2c\ + 2c2 — 3c3 = 0. 

By using Gaussian élimination, we find that the solution to this System is c\ — 

(— |) C3, C2 = — C3, C3 arbitrary. Thus, choosing C3 = 2, we obtain ci = —5, C2 = 

—2, C3 = 2 as a particular nonzero set of constants that satisfies (8); hence, the 
original vectors are linearly dépendent. ■ 


Example 4 Is the set 



T 


i 

' r) 

i 


1 

1 

U) < 


2 

•> 

7 

5 

L fil 


linearly independent? 

Solution Consider the vector équation 



T 


"5" 


"-3" 


" 0 " 

c\ 

2 

+ Q 

1 _ 

+ ^3 

1 

— 

0 


This is équivalent to the System 


ci + 5 c2 - 3c3 = 0, 

2ci + 7 c2 + C3 = 0. 

By using Gaussian élimination, we find that the solution to this System is ci = 
(— 26/3)c3, C2 = (7/3)c3, C3 arbitrary. Hence a particular nonzero solution is found 
by choosing C3 = 3; then ci = —26, C2 = 7, and, therefore, the vectors are linearly 
dépendent. ■ 

We conclude this section with a few important theorems on linear indepen- 
dence and dependence. 

Theorem 1 A set of vectors is linearly dépendent if and only ifone ofthe vectors 
is a linear combination ofthe others. 

Proof. Let {Vi,V2,...,V„}bea linearly dépendent set. Then there exist scalars 
ci, C2, . . . , c n , not ail zéro, such that (7) is satisfied. Assume ci 0. (Since at least 
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one of the c’s must differ from zéro, we lose no generality in assuming it is ci). 
Equation (7) can be rewritten as 

c 1 \ 1 = -C2V2 C3V3 c n \ n , 

or as 

Vi = — — v 2 — — v 3 -v„. 

ci ci ci 

Thus, Vi is a linear combination of V2, V3, . . . , V n . To complété the proof, we 
must show that if one vector is a linear combination of the others, then the set is 
linearly dépendent. We leave this as an exercise for the student (see Problem 36.) 

Observation 1 In order for a set of vectors to be linearly dépendent, it is not 
necessary for every vector to be a linear combination of the others, only that there 
exists one vector that is a linear combination of the others. For example, consider 
the vectors [1 0], [2 0], [0 1]. Here, [0, 1] cannot be written as a linear 

combination of the other two vectors; however, [2 0] can be written as a linear 
combination of [1 0] and [0 1], namely, [2 01 = 2[1 0] + 0[0 1]]; hence, the 

vectors are linearly dépendent. □ 

Theorem 2 The set consisting of the single vector Vi is a linearly independent 
set if and only if Y 1 7^ 0. 

Proof. Consider the équation ciVi = 0 . If Vi 7^ 0 , then the only way this équa- 
tion can be valid is if ci = 0; hence, the set is linearly independent. If Vi = 0 , then 
any ci 7^ 0 will satisfy the équation; hence, the set is linearly dépendent. □ 

Theorem 3 Any set of vectors that contains the zéro vector is linearly dépendent. 

Proof Consider the set {Vi, V2, . . . , \ n , 0 }. Pickci = C2 = • • • = c n = 0, c w +i = 
5 (any other number will do). Then this is a set of scalars, not ail zéro, such that 


ciVi + C2Y 2 + • • * + c n \ n + c„ + i0 — 0; 

hence, the set of vectors is linearly dépendent. □ 

Theorem 4 If a set of vectors is linearly independent, any subset ofthese vectors 
is also linearly independent. 

Proof See Problem 37. □ 

Theorem 5 If a set of vectors is linearly dépendent, then any larger set, containing 
this set, is also linearly dépendent. 


Proof See Problem 38. 


□ 
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Problems 2.5 

In Problems 1 through 19, détermine whether or not the given set is linearly inde- 
pendent. 

1 . {[1 0 ], [0 1 ]}. 2 . {[1 1],[1 - 1 ]}. 


3. {[2 —4], [—3 6]}. 4. {[1 3], [2 -1], [1 1]). 



13. 5 , 0 , 1 . 14. {[1 1 0],[1 -1 0]}. 

|_ij [2] |_ij 


15. {[1 2 3], [-3 -6 -9]}. 

16. {[10 20 20], [10 -10 10], [10 20 10]}. 

17. {[10 20 20], [10 -10 10], [10 20 10], [20 10 20]}. 

18. {[2 1 1], [3 -1 4], [1 3 -2]}. 



20. Express the vector 
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as a linear combination of 


r 

"1 


1 


"1 



1 

1 

0 

•> 

1 



1 

O 
1 


-1 


1 



21. Can the vector [2 3] be expressed as a linear combination of the vectors 
given in (a) Problem 1, (b) Problem 2, or (c) Problem 3? 

22. Can the vector [1 1 1] T be expressed as a linear combination of the vectors 

given in (a) Problem 7, (b) Problem 8, or (c) Problem 9? 

23. Can the vector [2 0 3] T be expressed as a linear combination of the vectors 
given in Problem 8? 

24. A set of vectors S is a spanning set for another set of vectors R if every vector 
in R can be expressed as a linear combination of the vectors in S. Show that 
the vectors given in Problem 1 are a spanning set for ail two-dimensional row 
vectors. Hint: Show that for any arbitrary real numbers a and b , the vector 
[a b] can be expressed as a linear combination of the vectors in Problem 1. 

25. Show that the vectors given in Problem 2 are a spanning set for ail two- 
dimensional row vectors. 

26. Show that the vectors given in Problem 3 are not a spanning set for ail two- 
dimensional row vectors. 

27. Show that the vectors given in Problem 3 are a spanning set for ail vectors of 
the form [a —2a], where a désignâtes any real number. 

28. Show that the vectors given in Problem 4 are a spanning set for ail two- 
dimensional row vectors. 

29. Détermine whether the vectors given in Problem 7 are a spanning set for ail 
three-dimensional column vectors. 

30. Détermine whether the vectors given in Problem 8 are a spanning set for ail 
three-dimensional column vectors. 

31. Détermine whether the vectors given in Problem 8 are a spanning set for 
vectors of the form [a 0 a] J , where a dénotés an arbitrary real number. 

32. A set of vectors S is a basis for another set of vectors R if S is a spanning set 
for R and S is linearly independent. Détermine which, if any, of the sets given 
in Problems 1 through 4 are a basis for the set of ail two dimensional row 
vectors. 

33. Détermine which, if any, of the sets given in Problems 7 through 12 are a basis 
for the set of ail three dimensional column vectors. 

34. Prove that the columns of the 3x3 identity matrix form a basis for the set of 
ail three dimensional column vectors. 
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35. Prove that the rows of the 4x4 identity matrix form a basis for the set of ail 
four dimensional row vectors. 

36. Finish the proof of Theorem 1. ( Hint : Assume that Vi can be written as a 
linear combination of the other vectors.) 

37. Prove Theorem 4. 

38. Prove Theorem 5. 

39. Prove that the set of vectors {x, kx) is linearly dépendent for any choice of the 
scalar k. 

40. Prove that if x and y are linearly independent, then so too are x + y and x — y. 

41. Prove that if the set {xi, X 2 , . . . , x w } is linearly independent then so too is the 
set {k\xi, &2X2, . . . , k n x n ] for any choice of the non-zero scalars k\, k 2 , . . . ,k n . 

42. Let A be an n x n matrix and let {xi, X 2 , . . . , x^} and {yi, y 2 , . . . , y&} be two 
sets of ft-dimensional column vectors having the property that A x/ = y/ = 
1, 2, . . . , k. Show that the set {xi, X 2 , . . . , x*} is linearly independent if the set 

{yi,y2, ---.y*} is. 


2.6 Rarik 

If we interpret each row of a matrix as a row vector, the elementary row operations 
are precisely the operations used to form linear combinations; namely, multiplying 
vectors (rows) by scalars and adding vectors (rows) to other vectors (rows). This 
observation allows us to develop a straightforward matrix procedure for deter- 
mining when a set of vectors is linearly independent. It rests on the concept of 
rank. 

Définition 1 The row rank of a matrix is the maximum number of linearly inde- 
pendent vectors that can be formed from the rows of that matrix, considering each 
row as a separate vector. Analogically, the column rank of a matrix is the maximum 
number of linearly independent columns, considering each column as a separate 
vector. 

Row rank is particularly easy to détermine for matrices in row-reduced form. 

Theorem 1 The row rank of a row-reduced matrix is the number of nonzero 
rows in that matrix. 

Proof. We must prove two facts: First, that the nonzero rows, considered as 
vectors, form a linearly independent set, and second, that every larger set is linearly 
dépendent. Consider the équation 


c\V\ + C 2 \ 2 H E C r \ r = 0, 


( 9 ) 
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where vi is the first nonzero row, V 2 is the second nonzero row, . . . , and v r is the 
last nonzero row of a row-reduced matrix. The first nonzero element in the first 
nonzero row of a row-reduced matrix must be unity. Assume it appears in column 
j. Then, no other rows hâve a nonzero element in that column. Consequently, 
when the left side of Eq. (9) is computed, it will hâve c\ as its yth component. 
Since the right side of Eq. (9) is the zéro vector, it follows that c\ — 0. A similar 
argument then shows iteratively that C 2 , . . . , c r , are ail zéro. Thus, the nonzero 
rows are linearly independent. 

If ail the rows of the matrix are nonzero, then they must comprise a maximum 
number of linearly independent vectors, because the row rank cannot be greater 
than the number of rows in the matrix. If there are zéro rows in the row-reduced 
matrix, then it follows from Theorem 3 of Section 2.5 that including them could 
not increase the number of linearly independent rows. Thus, the largest number 
of linearly independent rows cornes from including just the nonzero rows. □ 


Example 1 Détermine the row rank of the matrix 



"1 0 
0 0 
0 0 
0 0 


-2 

1 

0 

0 


5 3" 
—4 1 
1 0 
0 0 


Solution A is in row-reduced form. Since it contains three nonzero rows, its row 
rank is three. ■ 


The following two theorems, which are proved in the Final Comments to this 
chapter, are fundamental. 

Theorem 2 The row rank and column rank of a matrix are equal. 

For any matrix A, we call this common number the rank of A and dénoté it by 
K A). 

Theorem 3 If B is obtainedfrom A by an elementary row ( or column ) operation, 
then r(B) = r(A). 

Theorems 1 through 3 suggest a useful procedure for determining the rank of 
any matrix: Simply use elementary row operations to transform the given matrix 
to row-reduced form, and then count the number of nonzero rows. 


Example 2 Détermine the rank of 


1 3 4 

2 -1 1 

3 2 5 

5 15 20 


A = 






80 


Chapter 2 Simultaneous Linear Equations 


Solution In Example 2 of Section 2.3, we transferred this matrix into the row- 
reduced form 


1 3 4 
0 1 1 
0 0 0 • 

0 0 0 

This matrix has two nonzero rows so its rank, as well as that of A, is two. ■ 


Example 3 Détermine the rank of 




Solution In Example 3 of Section 2.3, we transferred this matrix into the row- 
reduced form 


12 13 
0 1 3 12 . 

0 0 15 

This matrix has three nonzero rows so its rank, as well as that of B, is three. ■ 

A similar procedure can be used for determining whether a set of vectors is 
linearly independent: Form a matrix in which each row is one of the vectors in the 
given set, and then détermine the rank of that matrix. If the rank equals the num- 
ber of vectors, the set is linearly independent; if not, the set is linearly dépendent. 
In either case, the rank is the maximal number of linearly independent vectors 
that can be formed from the given set. 


Example 4 Détermine whether the set 



2 


"3" 


8" 



6 

? 

1 

» 

16 



-2 


2 


-3 



is linearly independent. 

Solution We consider the matrix 

"2 6 - 2 " 

3 1 2 . 

8 16 -3 
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Reducing this matrix to row-reduced form, we obtain 

"1 3 -1" 

0 1 -I ■ 

_° 0 °_ 

This matrix has two nonzero rows, so its rank is two. Since this is less than the 
number of vectors in the given set, that set is linearly dépendent. 

We can say even more: The original set of vectors contains a subset of two 
linearly independent vectors, the same number as the rank. Also, since no row 
interchanges were involved in the transformation to row-reduced form, we can 
conclude that the third vector is linear combination of the first two. ■ 


Example 5 Détermine whether the set 

{[0 1 2 3 0], [13-12 1], 

[2 6 -1 -3 1], [4 0 1 0 2]} 

is linearly independent. 

Solution We consider the matrix 

"0 1 2 3 0' 

13-1 2 1 

2 6-1-31’ 

4 0 1 0 2 

which can be reduced (after the first two rows are interchanged) to the row- 
reduced form 

"13-1 2 1" 

0 12 3 0 

0 0 1 -7 -1 * 

_0 0 0 1 

This matrix has four nonzero rows, hence its rank is four, which is equal to the 
number of vectors in the given set. Therefore, the set is linearly independent. ■ 



Example 6 Can the vector 
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be written as a linear combination of the vectors 


Solution The matrix 




? 



6 

4 


can be transformée! into the row-reduced form 

"3 6" 

0 0_|’ 

which has rank one; hence A has just one linearly independent row vector. In 
contrast, the matrix 



1 

6 

4 


can be transformed into the row-reduced form, 

"1 1 " 

0 1 , 

0 0 


which has rank two; hence B has two linearly independent row vectors. Since B 
is precisely A with one additional row, it follows that the additional row [1, 1] T 
is independent of the other two and, therefore, cannot be written as a linear 
combination of the other two vectors. ■ 


We did not hâve to transform B in Example 6 into row-reduced form to déter- 
mine whether the three-vector set was linearly independent. There is a more direct 
approach. Since B has only two columns, its column rank must be less than or equal 
to two (why?). Thus, the column rank is less than three. It follows from Theorem 
3 that the row rank of B is less than three, so the three vectors must be linearly 
dépendent. Generalizing this reasoning, we deduce one of the more important 
results in linear algebra. 

Theorem 4 In an n-dimensional vector sp ace, every set ofn + 1 vectors is linearly 
dépendent. 
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Problems 2.6 


In Problems 1-5, find the rank of the given matrix. 



1 2 0 
3 1-5' 




1 7 0 
0 11. 
1 1 0 



4 1 
2 3 . 
2 2 



12 4 2 
113 2 . 
12 4 2 


In Problems 6 through 22, use rank to détermine whether the given set of vectors 
is linearly independent. 

6. {[1 0], [0 1]}. 7. {[1 1], [1 -1]}. 


8. {[2 -4], [-3 6]}. 


9. {[1 3], [2 -1], [1 1]}. 







' 

"1 


"r 


1 

0 

1 



0 

5 

i 

1 

1 


. 

1 


i 

O 
1 


1 

I— 1 ‘ 

1 



r 

“1 


"1 


1 



0 

•> 

1 

•> 

-1 


. 

1 


1 


1 



r 

"1 


"3 


"2 



2 

•> 

2 

•> 

1 


. 

3 


i 

i 


3 



ii. 


1 


T 


T 



-1 

1 

1 

•> 

2 



■ 

V 


~r 


"2 

— i 

13. 


0 

•> 

0 

•> 

0 



■ 

1 


2_ 


1 

- 


' 

"0" 


"3“ 


"2 

— i 

15. 


0 

•> 

2 

•> 

1 




0 


1 


3 





"1 


"3 


"2 


"-1 


17. 


2 

•> 

2 

•> 

1 

•> 

2 




3 


1 


3 


-3 



18. {[1 1 0],[1 -1 0]}. 

19. {[1 2 3], [-3 -6 -9]}. 

20. {[10 20 20 ], [10 -10 10 ], [10 20 10 ]}. 

21 . {[10 20 20 ], [10 -10 10 ], [10 20 10 ], [20 10 20 ]}. 

22. {[2 1 1], [3 -1 4], [1 3 -2]}. 


23. Can the vector [2 3] be expressed as a linear combination of the vectors 
given in (a) Problem 6, (b) Problem 7, or (c) Problem 8? 
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24. Can the vector [1 1 1] T be expressed as a linear combination of the vectors 

given in (a) Problem 12, (b) Problem 13, or (c) Problem 14? 

25. Can the vector [2 0 3] T be expressed as a linear combination of the vectors 
given in Problem 13? 

26. Can [3 7] be written as a linear combination of the vectors [ 1 2] and [3 2] ? 

27. Can [3 7] be written as a linear combination of the vectors [1 2] and [4 8] ? 

28. Find a maximal linearly independent subset of the vectors given in Problem 9. 

29. Find a maximal linearly independent subset of the vectors given in Problem 13. 

30. Find a maximal linearly independent subset of the set. 

[1 2 4 0], [2 4 8 0], [1 -1 0 1], [4 2 8 2], [4 -1 4 3]. 

31. What is the rank of the zéro matrix? 

32. Show r(A T ) = r(A). 


2.7 Theory of Solutions 

Consider once again the System Ax = b of m équations and n unknowns given in 
Eq. (2). Designate the/i columns of Aby the vectors Vi, V 2 , . . . , V^.Then Eq. (2) 
can be rewritten in the vector form 


JC 1 V 1 + V 2 V 2 H h x n \ n = b. 



Example 1 Rewrite the following System in the vector form (10): 


v — 2y -h 3z = 7, 
4x + 5y — 6z — 8. 


Solution 






Thus, finding solutions to (1) and (2) is équivalent to finding scalars 
x\,X2 , . . . , x n that satisfy (10). This, however, is asking precisely the question “Is 
the vector b a linear combination of Vi, V2, . . . , V n ?” If b is a linear combination 
of Vi,y 2 , V w , then there will exist scalars * 1 , X2, . . . , x n that satisfy (10) and 
the System is consistent. If b is not a linear combination of these vectors, that is, if b 
is linearly independent of the vectors Vi , V2 , • • . , V w , then no scalars x\ , X2 , . . . , x n 
will exist that satisfy (10) and the System is inconsistent. 

Taking a hint from Example 6 of Section 2.6, we hâve the following theorem. 
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Theorem 1 The System Ax = b is consistent if and only ifr( A) = r(A b ). 

Once a System is deemed consistent, the following theorem spécifiés the 
number of solutions. 

Theorem 2 If the System Ax = b is consistent and r(A) = k then the solutions are 
expressible in terms of n — k arbitrary unknowns (where n represents the number 
of unknowns in the System). 

Theorem 2 is almost obvious. To détermine the rank of A b , we must reduce 
it to row-reduced form. The rank is the number of nonzero rows. With Gaussian 
élimination, we use each nonzero row to solve for the variable associated with the 
first nonzero entry in it. Thus, each nonzero row defines one variable, and ail other 
variables remain arbitrary. 


Example 2 Discuss the solutions of the System 


* + y — z = 1, 
.r + y — z = 0. 


Solution 


"1 1 -1" 


ni 

a h 

"1 1 -1 1 " 

1 1 -1 

, b = 

o 

i 

, A b = 

11-10 


Here, r(A) = 1, r(A b ) = 2. Thus, r(A) r(A b ) and no solution exists. ■ 


Example 3 Discuss the solutions of the System 

v + y + w = 3, 
2x + 2y + 2w = 6, 

—x — y — w — —3. 

Solution 


A = 

1 

2 

1 1 
2 2 

, b = 

3" 

6 

, A b = 

1 

2 

1 1 3" 

2 2 6 


-1 

-1 -1 


-3 


-1 

-1 -1 -3 


Here r( A) = r(A b ) = 1; hence, the System is consistent. In this case, n — 3 and 
k — 1; thus, the solutions are expressible in terms of 3 — 1 = 2 arbitrary unknowns. 
Using Gaussian élimination, we find that the solution is x = 3 — y — w where y and 
w are both arbitrary. ■ 
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Example 4 Discuss the solutions of the System 

2x — 3y + z — — 1, 
x — y -\-2z— 2, 
2x + y — 3z — 3. 

Solution 


A = 

"2 -3 1 

1 -1 2 

, b = 

"-1 

2 

. A b = 

"2 -3 1 -1" 

1-12 2 


2 1 -3 


3 


2 1-33 


Here r( A) = r(A b ) = 3, hence the System is consistent. Since n — 3 and k = 3, the 
solution will be in n — k — 0 arbitrary unknowns. Thus, the solution is unique (none 
of the unknowns are arbitrary) and can be obtained by Gaussian élimination as 
x — y — 2, z — l. ■ 


Example 5 Discuss the solutions of the System 

x + y — 2z = 1, 
2x + y 4- z = 2, 
3x + 2y — z — 3, 
4x + 2y + 2z = 4. 

Solution 


"1 

1 

-2 


"1" 


"1 

1 

-2 

1" 

2 

1 

1 

, b = 

2 

a h 

2 

1 

1 

2 

3 

2 

-1 

3 

, A b = 

3 

2 

-1 

3 

4 

2 

2 


i 

i 


4 

2 

2 

4 


Here r(A) = r(A b ) = 2. Thus, the System is consistent and the solutions will be in 
terms of 3 — 2 = 1 arbitrary unknowns. Using Gaussian élimination, we find that 
the solution is x = 1 — 3z, y = 5z, and z is arbitrary. ■ 

In a consistent System, the solution is unique ifk = n.Ifk^=n, the solution will 
be in terms of arbitrary unknowns. Since these arbitrary unknowns can be chosen 
to be any constants whatsoever, it follows that there will be an infinité number of 
solutions. Thus, a consistent System will possess exactly one solution or an infinité 
number of solutions; there is no inbetween. 
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A homogeneous System of simultaneous linear équations has the form 


011*1 + 012*2 H h a\ n x n = 0, 

«21*1 + 022*2 H h 02rc*rc = 0, 



or the matrix form 

Ax = 0. (12) 

Since Eq. (12) is a spécial case of Eq. (2) with b = 0, ail the theory developed 
for the System Ax = b remains valid. Because of the simplified structure of a 
homogeneous System, one can draw conclusions about it that are not valid for a 
nonhomogeneous System. For instance, a homogeneous System is always consis- 
tent. To verify this statement, note that x\ = *2 = • • • = x n — 0 is always a solution 
to Eq. (12). Such a solution is called the trivial solution. It is, in general, the non- 
trivial solutions (solutions in which one or more of the unknowns is different from 
zéro) that are of the greatest interest. 

It follows from Theorem 2, that if the rank of A is less than n(n being the 
number of unknowns), then the solution will be in terms of arbitrary unknowns. 
Since these arbitrary unknowns can be assigned nonzero values, it follows that 
nontrivial solutions exist. On the other hand, if the rank of A equals n, then the 
solution will be unique, and, hence, must be the trivial solution (why?). Thus, it 
follows that: 

Theorem 3 The homogeneous System (12) will admit nontrivial solutions if and 
only if r( A) 7 ^ n. 


Problems 2.7 

In Problems 1-9, discuss the solutions of the given System in terms of consistency 
and number of solutions. Check your answers by solving the Systems wherever 
possible. 

1. * — 2y = 0, 2. v + y = 0, 

x + y = 1 , 2x — 2y = 1 , 

2x — y — 1. x— y — 0. 

3. x + y + Z = 

* - y + z = 

3x + y + 3z = 


1, 4 . x + 3 y + 2z — vu 

2 , 2x — y + z-\- vu 

4. 


2 , 

3. 
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5 . 2x — y + 

z 

— 

0, 

x + 2y - 

Z 

— 

4, 

x + y + 

Z 

— 

1. 

7 . x — y 

+ 

2z 

= 0, 

2x + 3y 

— 

z 

= 0, 

— 2x + 7y 

— 

7z 

= 0. 

9 . x — 2y + 

3z 

+ 

3 w = 

y - 

2z 

+ 

2 w — 

x + y 

3z 

+ 

9 w = 


6 . 2x + 3y = 0, 
x — 4y = 0, 


x — 

y + 

2 z 

= 0, 

2x — 

3 y - 

z 

= 0, 

— 2x + 

ly - 

9z 

= 0. 


0 , 

0 , 

0 . 


2.8 Final Comments on Chapter 2 

We would like to show that the column rank of a matrix equals its row rank, and 
that an elementary row operation of any kind does not alter the rank. 

Lemma 1 If B is obtained from A by interchanging two columns of A, then both 
A and B hâve the same column rank. 

Proof. The set of vectors formed from the columns of A is identical to the set 
formed from the columns of B, and, therefore, the two matrices must hâve the 
same column rank. □ 

Lemma 2 If Ax = 0 and Bx = 0 hâve the same set of solutions, then the column 
rank of A. is less than or equal to the column rank of B. 

Proof. Let the order of A be m x n. Then, the System Ax = 0 is a set of m 
équations in the n unknowns x \ , X 2 , . . . , x n , which has the vector form 


xi Ai + X 2 A 2 H h x n A n = 0, (13) 

where Ai, A 2 , . . . , A n dénoté the columns of A. Similarly, the System Bx = 0 has 
the vector form 


xiBi + X 2 B 2 + • • • + x n B n — 0. (14) 

We shall assume that the column rank of A is greater than the column rank of B 
and show that this assumption leads to a contradiction. It will then follow that the 
reverse must be true, which is precisely what we want to prove. 

Dénoté the column rank of A as a and the column rank of B as b. We assume 
that a > b. Since the column rank of A is a , there must exist a columns of A that 
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are linearly independent. If these columns are not the first a columns, rearrange 
the order of the columns so they are. Lemma 1 guarantees such reorderings do 
not alter the column rank. Thus, Ai, A 2 , . . . , A fl are linearly independent. Since 
a is assumed greater than b , we know that the first a columns of B are not lin- 
early independent. Since they are linearly dépendent, there must exist constants 
ci, C 2 , . . . , c a — not ail zéro — such that 


ciBi + C 2 B 2 + • • • + c fl B fl — 0. 


It then follows that 


ciBi -|- C 2 B 2 + • • • + OfBtf + OBfl+i H- • • • H- 0B W — 0, 
from which we conclude that 


X\ — Cl, X2 — C2, . . . , X a — C a , %ci + 1 — 



is a solution of Eq. (14). Since every solution to Eq. (14) is also a solution to Eq. 
(12), we hâve 


ci Ai H- C 2 A 2 + • • • + c a Aa H- 0A o _|_i H- • • • H- 0A /7 — 0 , 


or more simply 


ciAi + C 2 A 2 + • • • + c a A a — 0, 


where ail the c’s are not ail zéro. But this implies that the first a columns of A 
are linearly dépendent, which is a contradiction of the assumption that they were 
linearly independent. □ 

Lemma 3 If Ax = 0 and Bx = 0 hâve the same set of solutions, then A and B 
hâve the same column rank. 

Proofi If follows from Lemma 2 that the column rank of A is less than or equal 
to the column rank of B. By reversing the rôles of A and B, we can also conclude 
from Lemma 2 that the column rank of B is less than or equal to the column rank 
of A. As a resuit, the two column ranks must be equal. □ 

Theorem 1 An elementary row operation does not alter the column rank of a 
matrix. 

Proof. Dénoté the original matrix as A, and let B dénoté a matrix obtained 
by applying an elementary row operation to A; and consider the two homoge- 
neous Systems Ax = 0 and Bx = 0. Since elementary row operations do not alter 
solutions, both of these Systems hâve the same solution set. Theorem 1 follows 
immediately from Lemma 3. □ 
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Lemma 4 The column rank of a matrix is less than or equal to its row rank. 

Proof. Dénoté rows of A by Ai, A2, . . . A m , the column rank of matrix A by c 
and its row rank by r. There must exist r rows of A which are linearly independent. 
If these rows are not the first r rows, rearrange the order of the rows so they are. 
Theorem 1 guarantees such reorderings do not alter the column rank, and they 
certainly do not alter the row rank. Thus, Ai , A2, . . . , A r are linearly independent. 
Define partitioned matrices R and S by 



"Af 



A r+ i 

R = 

A2 

and 

s = 

A r+ 2 

• 


A, 



A77 


Then A has the partitioned form 



Every row of S is a linear combination of the rows of R. Therefore, there exist 
constants tq such that 


A r+ 1 — f r +l,lAi 4- ^r+1,2 A2 + • • * + ^r+l,r A r , 
A r+ 2 = t r + 2,1 Al + t r + 2 , 2^2 H + t r + 2,/-A r , 


A n ~ t n , 1A1 + ^2,2 A2 + • * • + t nj A r , 

which may be written in the matrix form 


where 


T = 



S = TR, 


^•+ 1,1 

fr + 1,2 * ‘ ‘ 

tr +1 ,n 

^ r + 2,1 

tr+ 2,2 • ‘ ‘ 

• • 

tr+2,n 

• 

tn, 1 

• • 

tn, 2 

• 

hi,n 


Then, for any /i-dimensional vector x, we hâve 



R" 

Y 

Rx" 


Rx " 

S_ 

A. — 

Sx 


TRx 
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Thus, Ax = 0 if and only if Rx = 0. It folio ws from Lemma 3 that both A and R 
hâve the same column rank. But the columns of R are r-dimensional vectors, so 
its column rank cannot be larger than r. Thus, 

c = column rank of A = column rank of R < r = row rank of A □ 

Lemma 5 The row rank of a matrix is less than or equal to its column rank. 

Proof By applying Lemma 4 to A T , we conclude that the column rank of A T is 
less than or equal to the row rank of A T . But since the columns of A T are the rows 
of A and vice-versa, the resuit follows immediately. □ 

Theorem 2 The row rank of a matrix equals its column rank. 

Proof. The resuit is immédiate from Lemmas 4 and 5. □ 

Theorem 3 An elementary row operation does not alter the row rank of a matrix. 

Proof This theorem is an immédiate conséquence of both Theorems 1 and 2. 

□ 
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The Inverse 



3.1 Introduction 

Définition 1 An inverse of an n x n matrix A is a n x n matrix B having the 
property that 


AB = BA = I. (1) 

Here, B is called an inverse of A and is usually denoted by A -1 . If a square 
matrix A has an inverse, it is said to be invertible or nonsingular. If A does not 
possess an inverse, it is said to be singular. Note that inverses are only defined 
for square matrices. In particular, the identity matrix is invertible and is its own 
inverse because 


II = 1. 


Example 1 Détermine whether 


are inverses for 



ri p 


"-2 

1 " 

B = 

2 

1 1 

_ 3 4_ 

or C = 

3 

2 

1 

2_ 



2 

4 * 


Solution B is an inverse if and only if AB = BA = I; C is an inverse if and only 
if AC = CA = I. Here, 



"l 2 




ri p 


"l o" 

AB = 




— 





3 4 


i 

<— <ht 

r-H|m 

1 


13 5 

_ 3 2_ 


0 1 
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while 



’l 2 


i 

T— I 

(N 

1 

1 


i 

O 

t-H 

1 


"-2 l" 


' (N 

r-H 
1 

3 4 


3 1 

_ 2 2_ 


0 1 


3 1 

_ 2 2_ 


3 4 



Thus, B is not an inverse for A, but C is. We may write A 1 = C. ■ 


Définition 1 is a test for checking whether a given matrix is an inverse of another 
given matrix. In the Final Comments to this chapter we prove that if AB = I for 
two square matrices of the same order, then A and B commute, and BA = I. Thus, 
we can reduce the checking procedure by half. A matrix B is an inverse for a square 
matrix A if either AB = I or BA = I; each equality automatically guarantees the 
other for square matrices. We will show in Section 3.4 that an inverse is unique. If 
a square matrix has an inverse, it has only one. 

Définition 1 does not provide a method for finding inverses. We develop such 
a procedure in the next section. Still, inverses for some matrices can be found 
directly. 

The inverse for a diagonal matrix D having only nonzero éléments on its main 
diagonal is also a diagonal matrix whose diagonal éléments are the reciprocals of 
the corresponding diagonal éléments of D. That is, if 


then 



0 


0 





1 

^2 


1 



0 


I— _i 

It is easy to show that if any diagonal element in a diagonal matrix is zéro, then 
that matrix is singular. (See Problem 57.) 

An elementary matrix E is a square matrix that generates an elementary row 
operation on a matrix A (which need not be square) under the multiplication 
EA. Elementary, matrices are constructed by applying the desired elementary 
row operation to an identity matrix of appropriate order. The appropriate order 
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for both I and E is a square matrix having as many columns as there are rows 
in A; then, the multiplication EA is defined. Because identity matrices contain 
many zéros, the process for constructing elementary matrices can be simplified 
still further. After ail, nothing is accomplished by interchanging the positions of 
zéros, multiplying zéros by nonzero constants, or adding zéros to zéros. 

(i) To construct an elementary matrix that interchanges the /th row with the j th 
row, begin with an identity matrix of the appropriate order. First, interchange 
the unity element in the i — i position with the zéro in the j — i position, and 
then interchange the unity element in the j — j position with the zéro in the 
/ — j position. 

(ii) To construct an elementary matrix that multiplies the /th row of a matrix by 
the nonzero scalar k , replace the unity element in the i — i position of the 
identity matrix of appropriate order with the scalar k. 

(iii) To construct an elementary matrix that adds to the y th row of a matrix k times 
the /th row, replace the zéro element in the j — i position of the identity matrix 
of appropriate order with the scalar k. 


Example 2 Find elementary matrices that when multiplied on the right by any 
4x3 matrix A will (a) interchange the second and fourth rows of A, (b) multiply 
the third row of A by 3, and (c) add to the fourth row of A — 5 times its second row. 


Solution 



"1 

0 

0 

0 " 


"1 

0 

0 

0 " 


"1 

0 

0 

0 " 

0 

0 

0 

1 

. (b) 

0 

1 

0 

0 

, (<0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

3 

0 

0 

0 

1 

0 

_0 

1 

0 

0 _ 


_0 

0 

0 

1 _ 


_0 

-5 

0 

1 _ 


Example 3 Find elementary matrices that when multiplied on the right by any 
3x5 matrix A will (a) interchange the first and second rows of A, (b) multiply the 
third row of A by —0.5, and (c) add to the third row of A — 1 times its second row. 

Solution 



"0 1 0“ 


"10 0 “ 


"1 0 0” 


(a) 

1 

O 

O O 

o 

i 

, (b) 

0 1 0 
_0 0 — 0.5_ 

, (c) 

0 1 0 
_0 -1 1_ 

. ■ 


The inverse of an elementary matrix that interchanges two rows is the matrix 
itself, it is its own inverse. The inverse of an elementary matrix that multiplies 
one row by a nonzero scalar k is gotten by replacing k by 1/k. The inverse of 
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an elementary matrix which adds to one row a constant k times another row is 
obtained by replacing the scalar k by —k. 


Example 4 Compute the inverses of the elementary matrices found in 
Example 2. 

Solution 



Example 5 Compute the inverses of the elementary matrices found in 
Example 3. 


Solution 

"0 1 ol [10 ol [1 0 0“ 

(a) 1 0 0 , (b) 0 1 0 , (c) 0 1 0 . 

0 0 1 0 0 -2 0 1 1 

Finally, if A can be partitioned into the block diagonal form, 



then A is invertible if and only if each of the diagonal blocks Ai, A 2 , . . . , A„ is 
invertible and 
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Example 6 Find the inverse of 

"2 0 0 0 0 0 0 " 

0 5 0 0 0 0 0 

0 0 1 0 0 0 0 

A = 0 0 4 1 0 0 0. 

0 0 0 0 1 0 0 

0 0 0 0 0 0 1 

0 0 0 0 0 1 0 


Solution Set 



and A 3 


0 1 



•> 


then, A is in the block diagonal form 



Here Ai is a diagonal matrix with nonzero diagonal éléments, A 2 is an elementary 
matrix that adds to the second row four times the first row, and A 3 is an elementary 
matrix that interchanges the second and third rows; thus 




and 



0 1 



» 


and 



0 0 0 0 0 0 “ 

0 \ 0 0 0 0 0 

0 0 1 0 0 0 0 

0 0 -4 1 0 0 0 • ■ 

0 0 0 0 1 0 0 

0 0 0 0 0 0 1 

0 0 0 0 0 1 0 
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Problems 3.1 


1. Détermine if any of the following matrices are inverses for 



2. Détermine if any of the following matrices are inverses for 


(a) 

(c) 



3. Calculate directly the inverse of 



8 2 
5 3 ‘ 


Hint : Define 



b 

d ‘ 


Calculate AB, set the product equal to I, and then solve for the éléments of B. 

4. Use the procedure described in Problem 3 to calculate the inverse of 




5. Use the procedure described in Problem 3 to calculate the inverse of 


D = 


1 1 ' 
1 1 
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6 . Show directly that the inverse of 


A = 


a b 
c d 


when a d — bc ^ 0 is 


A -1 = 


1 


ad — bc 


d —b 
—c a 


7. Use the results of Problem 6 to calculate the inverse of 

1 1 
3 4 

8. Use the results of Problem 6 to calculate the inverse of 


2 1 
4 3 


9. Use the results of Problem 6 to calculate the inverse of 


1 - 
1 2 

1 1 

L2 3 J 


10. Use the results of Problem 6 to calculate the inverse of 

10 20 ' 

30 40 


In Problems 11 through 26, find elementary matrices that when multiplied on the 
right by a matrix A will generate the specified resuit. 

11. Interchange the order of the first and second row of the 2x2 matrix A. 

12. Multiply the first row of a 2 x 2 matrix A by three. 

13. Multiply the second row of a 2 x 2 matrix A by — 5. 

14. Multiply the second row of a 3 x 3 matrix A by — 5. 

15. Add to the second row of a 2 x 2 matrix A three times its first row. 

16. Add to the first row of a 2 x 2 matrix A three times its second row. 

17. Add to the second row of a 3 x 3 matrix A three times its third row. 

18. Add to the third row of a 3 x 4 matrix A five times its first row. 

19. Add to the second row of a 4 x 4 matrix A eight times its fourth row. 

20. Add to the fourth row of a 5 x 7 matrix A —2 times its first row. 

21. Interchange the second and fourth rows of a 4 x 6 matrix A. 
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22. Interchange the second and fourth rows of a 4 x 4 matrix A. 

23. Interchange the second and fourth rows of a 6 x 6 matrix A. 

24. Multiply the second row of a 2 x 5 matrix A by seven. 

25. Multiply the third row of a 5 x 2 matrix A by seven. 

26. Multiply the second row of a 3 x 5 matrix A by —0.2. 


In Problems 27 through 42, find the inverses of the given elementary matrices. 




28. 


1 2 ' 
0 1 






5 



r-H 
1 

0 

i 

o 


1 

O 

1 

0" 

31. 

0 

2 

0 

, 32. 

1 

0 

0 


1 

O 

0 

1 


1 

O 

0 

1 



1 0 0 
0 10 , 
3 0 1 



1 

0 

0 


0 3 

1 0 , 
0 1 



"1 

0 

1 

O 


"1 

0 

0" 

35. 

0 

1 

-2 

, 36. 

0 

1 

0 


1 

O 

0 

1 


1 

O 

0 

-4 




"1 

0 

0 

0 


"1 

0 

0 

0 


1 

0 

0 

0 

0 

1 

0 

0 

Ui 

00 

• 

0 

1 

0 

7 

, 39. 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

-3 

0 

1 

0 

0 

0 

1 

0 


0 

0 

0 

1 


0 

0 

0 

1 

"0 

0 

0 

r 


"1 

0 

0 

0" 


"1 

0 

0 

0“ 

0 

1 

0 

0 

41. 

0 

1 

0 

0 

42. 

0 

1 

0 

0 

0 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

2 

0 

1 

0 

0 

0 


0 

0 

0 

1 


_0 

0 

0 

1_ 


In Problems 43 through 55, find the inverses, if they exist, of the given diagonal or 
block diagonal matrices. 



"2 0 " 


"-1 0 " 


"3 0 " 


T o " 

43. 

0 3 

44. 

0 0 

45. 

_0 — 3 _ 

, 46. 

2 

0 — - 
L 3 J 



10 

0 

0 



1 

1 

0 



-4 

0 

0 

47. 

0 

5 

0 

•> 

48. 

0 

1 

0 

1 

49. 

0 

-2 

0 


0 

0 

5 



0 

0 

-1 



0 

0 

3 

5 J 


"1 

2 

0 

0" 


"2 

0 

0 

0" 


“4 0 

0 

0" 

50. 

0 

1 

0 

0 

51. 

0 

3 

0 

0 

52. 

0 5 

0 

0 


0 

0 

1 

0 


0 

0 

1 - 

-3 

^ Mm • 

0 0 

6 

0 


0 

0 

2 

1 


0 

0 

0 

1 


0 0 

0 

1 
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"0 

1 

0 

0“ 


"0 

0 

1 

0“ 


“4 

0 

0 

0" 

1 

0 

0 

0 

, 54. 

0 

1 

0 

0 

, 55. 

0 

5 

0 

0 

0 

0 

0 

1 

7 

1 

0 

0 

0 


0 

0 

1 

6 

0 

0 

1 

0 


0 

0 

0 

7 


0 

0 

0 

1 


56. Prove that a square zéro matrix does not hâve an inverse. 

57. Prove that if a diagonal matrix has at least one zéro on its main diagonal, then 
that matrix cannot hâve an inverse. 

58. Prove that if A 2 = I, then A -1 = A. 


3.2 Calculating Inverses 

In Section 2.3, we developed a method for transforming any matrix into row- 
reduced form using elementary row operations. If we now restrict our attention 
to square matrices, we may say that the resulting row-reduced matrices are upper 
triangular matrices having either a unity or zéro element in each entry on the 
main diagonal. This provides a simple test for determining which matrices hâve 
inverses. 

Théo rem 1 A square matrix has an inverse if and only if réduction to row- 
reduced form by elementary row operations results in a matrix having ail unity 
éléments on the main diagonal. 

We shall prove this theorem in the Final Comments to this chapter as 

Theorem 2 An n x n matrix has an inverse if and only if it has rank n. 

Theorem 1 not only provides a test for determining when a matrix is invertible, 
but it also suggests a technique for obtaining the inverse when it exists. Once a 
matrix has been transformed to a row-reduced matrix with unity éléments on the 
main diagonal, it is a simple matter to reduce it still further to the identity matrix. 
This is done by applying elementary row operation (E3) — adding to one row of 
a matrix a scalar times another row of the same matrix— to each column of the 
matrix, beginning with the last column and moving sequentially toward the first 
column , placing zéros in ail positions above the diagonal éléments. 


Example 1 Use elementary row operations to transform the upper triangular 
matrix 



1 2 
0 1 
0 0 


1 

3 

1 


to the identity matrix. 
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Solution 


1 

2 

h - 1 

1 


"1 

2 

1 


0 

1 

3 

-> 

0 

1 

0 

* 

0 

0 

1 


0 

0 

1 





"1 

2 

0“ 




-> 

0 

1 

0 

* 




1 

0 

0 

1 





"1 

0 

0 " 




-> 

0 

1 

0 

* 




0 

0 

1 



by adding to 

the second row (—3) 

times the third row 

by adding to 
the first row (—1) 
times the third row 

by adding to 

the first row (—2) ■ 

times the second row 


To summarize, we now know that a square matrix A has an inverse if and only 
if it can be transformed into the identity matrix by elementary row operations. 
Moreover, it follows from the previous section that each elementary row operation 
is represented by an elementary matrix E that generates the row operation under 
the multiplication EA. Therefore, A has an inverse if and only if there exist a 
sequence of elementary matrices. Ei, E2, . . . , E k such that 

E/:E^_i • • • E3 E2 Ei A = I. 

But, if we dénoté the product of these elementary matrices as B, we then hâve 
B A = I, which implies that B = A -1 . That is, the inverse of a square matrix A of 
full rank is the product of those elementary matrices that reduce A to the identity 
matrix! Thus, to calculate the inverse of A, we need only keep a record of the 
elementary row operations, or equivalently the elementary matrices, that were 
used to reduce A to I. This is accomplished by simultaneously applying the same 
elementary row operations to both A and an identity matrix of the same order, 
because if 


E k Efc-1 • • • E3 E2 Ei A = I, 


then 


(E* E^-i • • • E3 E2 Ei)I = E* Ek~i • • • E3 E2 Ei = A 1 . 

We hâve, therefore, the following procedure for calculating inverses when they 
exist. Let A be the n x n matrix we wish to invert. Place next to it another n x n 
matrix B which is initially the identity. Using elementary row operations on A, 
transform it into the identity. Each time an operation is performed on A, repeat 
the exact same operation on B. After A is transformed into the identity, the matrix 
obtained from transforming B will be A -1 . 

If A cannot be transformed into an indentity matrix, which is équivalent to 
saying that its row-reduced from contains at least one zéro row, then A does not 
hâve an inverse. 
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Example 2 Invert 



2 

4 * 


Solution 


1 2 
3 4 


1 

0 



ri 

2 

1 

o" 

i — 

O 

-2 

-3 

ij 


"l 2 

1 0“ 

0 1 

3 1 

2 2_ 


by adding to 

the second row (—3) 

times the first row 

, 

by multiplying 

the second row by (— \) 


A has been transformed into row-reduced form with a main diagonal of only unity 
éléments; it has an inverse. Continuing with transformation process, we get 

by adding to 
the first row (—2) 
times the second row 


1 0 
0 1 


-2 

3 

2 


1 

2 


Thus, 



Example 3 Find the inverse of 



5 8 
0 2 
4 3 



Solution 


5 

8 

1 

1 

0 

0 " 


"1 

1.6 

0.2 

0.2 

0 

0 " 


0 

2 

1 

0 

1 

0 

-* 

0 

2 

1 

0 

1 

0 


4 

3 

-1 

0 

0 

1 


4 

3 

-1 

0 

0 

1 

■ 







"1 

1.6 

0.2 

0.2 

0 

0 







-* 

0 

2 

1 

0 

1 

0 

* 







0 

- 3.4 

- 1.8 

- 0.8 

0 

1 



by multiplying the 
first row by (0.2) 


by adding to the 
third row (—4) 
times the first row 





104 


Chapter 3 The Inverse 


1 

1.6 

0.2 

0.2 

0 

0" 


0 

1 

0.5 

0 

0.5 

0 


0 

-3.4 

-1.8 

-0.8 

0 

1 



1 

1.6 

0.2 

0.2 

0 

0 


0 

1 

0.5 

0 

0.5 

0 

* 

0 

0 

-0.1 

-0.8 

1.7 

1 



by multiplying the 
second row by (0.5) 


by adding to the 
third row (3.4) 
times the second row 


"1 

1.6 

0.2 

0.2 

0 

0" 

0 

1 

0.5 

0 

0.5 

0 

0 

0 

1 

8 

-17 

-10 


f by multiplying the 
I third row by (—0.1) 


A has been transformed into row-reduced form with a main diagonal of only unity 
éléments; it has an inverse. Continuing with the transformation process, we get 


Thus, 


"1 

1.6 

0.2 

0.2 

0 

0 " 

0 

1 

0 

-4 

9 

5 

0 

0 

1 

8 

-17 

-10 


by adding to the 
second row (—0.5) 
times the third row 


"1 

1.6 

0 

-1.4 

3.4 

2" 

0 

1 

0 

-4 

9 

5 

0 

0 

1 

8 

-17 

-10 


by adding to the 
first row (—0.2) 
times the third row 


"1 

0 

0 

5 

-h 

-6" 

0 

1 

0 

-4 

9 

5 

0 

0 

1 

8 

-17 

-10 


by adding to the 
first row (—1.6) 
times the second row 



Example 4 Find the inverse of 



0 1 
1 1 
1 1 


1 

1 

3 


Solution 


0 

1 

1 

1 

0 

0 

1 

1 

1 

0 

1 

0 

1 

1 

3 

0 

0 

1 


1 

1 

1 

0 

1 

0 

0 

1 

1 

1 

0 

0 

1 

1 

3 

0 

0 

1 


f by interchanging the 
I first and second rows 
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by adding to the 
the third row (— 1) 
times the first row 

, 

by multiplying the 
third row by (^) 

by adding to the 
second row (—1) 
times the third row 

, 

by adding to the 
first row (—1) 
times the third row 

, 

by adding to the 
first row (—1) 
times the second row 

. 


Thus, 



Example 5 Invert 




Solution 


12 10 ] [12 10 " 
2 4 0 ij ^ L° 0 “ 2 1 * 


by adding to 

the second row (—2) 

times the first row 

, 


A has been transformed into row-reduced form. Since the main diagonal contains 
a zéro element, here in the 2-2 position, the matrix A does not hâve an inverse. It 
is singular. ■ 
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Problems 3.2 


In Problems 1-20, find the inverses of the given matrices, if they exist. 




2 . 


'2 1 
1 2 



4 4 
4 4 ’ 







5 



"1 

1 

i 

O 


1 

O 

0 

1 

7. 

1 

0 

1 

8. 

1 

0 

0 


1 

O 

1 

1 


1 

O 

1 

— i 

o 



2 0-1 
0 1 2 , 

3 1 1 



1 2 3 
4 5 6, 
7 8 9 



2 0 0 
5 10, 
4 1 1 



2 1 5 

0 3-1, 
0 0 2 



3 2 1 

4 0 1, 
3 9 2 



1 

2 

-1" 


"1 

2 

r 

14. 

2 

0 

1 

, 15. 

3 

-2 

-4 


-1 

1 

3 


2 

3 

-i 



"2 

4 

3 


"5 

0 

-1" 

16. 

3 

-4 

-4 

17. 

2 

-1 

2 


5 

0 

1 

i 


2 

3 

-1 



3 1 

1 3 

2 3 



5 



1112 
0 1-1 1 
0 0 2 3 ’ 

0 0 0 -2 



10 0 0 
2-10 0 
4 6 2 0 * 

3 2 4 -1 


21. Use the results of Problems 11 and 20 to deduce a theorem involving inverses 
of lower triangular matrices. 

22. Use the results of Problems 12 and 19 to deduce a theorem involving the 
inverses of upper triangular matrices. 

23. Matrix inversion can be used to encode and décodé sensitive messages for 
transmission. Initially, each letter in the alphabet is assigned a unique positive 
integer, with the simplest correspondence being 


ABCDEFGHI J KLMNOPQRS TUVWXYZ 

•l» .l» 'l* *1"" "l* '1* "l* "l* "l* '1* '1* '1* '1' * 


1 23 4 567 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 
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Zéros are used to separate words. Thus, the message 

SHE IS A SEER 


is encoded 


19 8 5 0 9 19 0 1 0 19 5 5 18 0. 

This scheme is too easy to decipher, however, so a scrambling effect is added 
prior to transmission. One scheme is to package the coded string as a set of 
2-tuples, multiply each 2-tuple by a 2 x 2 invertible matrix, and then transmit 
the new string. For example, using the matrix 



2 

3 


•> 


the coded message above would be scrambled into 


’l 2 


~19" 

2 3 


8 


35 
62 ’ 



2 

3 


5 

0 


5 

10 ’ 


’l 2 


" 9 


'47' 

2 3 


19 


75 


etc., 


and the scrambled message becomes 


35 62 5 10 47 75 


Note an immédiate benefit from the scrambling: the letter S, which was orig- 
inally always coded as 19 in each of its three occurrences, is now coded as a 
35 the first time and as 75 the second time. Continue with the scrambling, and 
détermine the final code for transmitting the above message. 

24 . Scramble the message SHE IS A SEER using, matrix 




25. Scramble the message AARON IS A NAME using the matrix and steps 
described in Problem 23. 
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26. Transmitted messages are unscrambled by again packaging the received mes- 
sage into 2-tuples and multiplying each vector by the inverse of A. To décodé 
the scrambled message 


18 31 44 72 


using the encoding scheme described in Problem 23, we first calculate 




and then 


'-3 2" 


'18' 

2 -1_ 


_31_ 




2 ' 

-1 


’44" 

72 


" 12 " 
16 ‘ 


The unscrambled message is 


8 5 12 16 


which, according to the letter-integer correspondence given in Problem 23, 
translates to HELP. Using the same procedure, décodé the scrambled message 

26 43 40 60 18 31 28 51. 


27. Use the decoding procedure described in Problem 26, but with the matrix A 
given in Problem 24, to decipher the transmitted message 

16 120 -39 131 -27 45 38 76 -51 129 28 56. 


28. Scramble the message SHE IS A SEER by packaging the coded letters into 
3-tuples and then multiplying by the 3x3 invertible matrix 



0 1 

1 1 . 

1 0 


Add as many zéros as necessary to the end of the message to generate complété 
3-tuples. 



3.3 Simultaneous Equations 
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3.3 Simultaneous Equations 

One use of the inverse is in the solution of Systems of simultaneous linear 
équations. Recall, from Section 1.3 that any such System may be written in the form 

Ax = b, (2) 

where A is the coefficient matrix, b is a known vector, and x is the unknown vector 
we wish to find. If A is invertible, then we can premultiply (2) by A -1 and obtain 

A _1 Ax = A -1 b. 


But A 1 A = 1, therefore 


Ix = A -1 b 


or 

x = A -1 b. (3) 

Hence, (3) shows that if A is invertible, then x can be obtained by premultiplying 
b by the inverse of A. 


Example 1 Solve the following System for x and y: 

x — 2 y — —9, 

— 3x + y = 2. 


Solution Define 



then the System can be written as Ax = b, hence x = A l b. Using the method 
given in Section 3.2 we find that 



Thus, 



Using the définition of matrix equality (two matrices are equal if and only if their 
corresponding éléments are equal), we hâve that x —1 and y = 5. ■ 
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Example 2 Solve the following System for x, y, and z: 


5 x + 8y + z = 2, 
2y + z = -1, 
4x + 3 y — z = 3. 


Solution 



00 

IT) 

1 

1 




r 2i 

A = 

0 2 
_4 3 

1 

-1 

, x = 

T 

z_ 

, b = 

i 

l 

LàJ b-L 

1 


A 1 is found to be (see Example 3 of Section 3.2) 

5 -11 -6" 

-4 9 5 . 

8 -17 -10 


Thus, 



X 


5 

-h 

-6" 


2 


3" 


y 

= x = A _1 b = 

-4 

9 

5 


-1 

— 

-2 


z_ 


8 

-17 

-10 


3 


3 

hence x = 3, 

y = 

: —2, and z = 3. 

■ 







Not only does the invertibility of A provide us with a solution of the System 
Ax = b, it also provides us with a means of showing that this solution is unique 
(that is, there is no other solution to the System). 


Theorem 1 If A is invertible, then the System of simultaneous linear équations 
given by Ax = b has one and only one solution. 

Proof. Define w = A -1 b. Since we hâve already shown that w is a solution to 
Ax = b, it follows that 


Aw = b. (4) 

Assume that there exists another solution y. Since y is a solution, we hâve that 

Ay = b. (5) 


Equations (4) and (5) imply that 


Aw = Ay. 


( 6 ) 
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Premultiply both sides of (6) by A 1 . Then 

A _1 Aw = A _1 Ay, 
Iw = Iy, 


or 


w = y. 

Thus, we see that if y is assumed to be a solution of Ax = b, it must, in fact, equal 
w. Therefore, w = A -1 b is the only solution to the problem. □ 

If A is singular, so that A -1 does not exist, then (3) is not valid and other 
methods, such as Gaussian élimination, must be used to solve the given System of 
simultaneous équations. 


Problems 3.3 


In Problems 1 through 12, use matrix inversion, if possible, to solve the given 


Systems of équations: 

' JL 

V—-' 

1 . x 2 y = —3, 
3x + y — 1. 

2. a + 2b — 5, 

—3a + b — 13. 


3. + 2 y = 6, 

2x — 3 y = 7. 

4. 4/ — p — 1, 

5/ — 2p — —1. 


5. 2x + 3y = 8, 
6x + 9y — 24. 

6. x-\-2y— z — — 1, 

2x + 3y + 2z — 5, 
y— z = 2. 


7. 2x + 3y — z = 4, 
—x — 2y + z = —2, 
3x — y —2. 

8. 60/ + 30m + 20 n = 
30/ + 20m + 15 n = - 
20/ + 15m + 12 n = - 

0, 

-10, 

-10. 

9. 2 r + 4 s — 2, 

3 r 2s -|- t — 8, 
5r — 3s + lt — 15. 

10 . 2r + 4s = 3, 

3r + 2s + t — 8, 
5r — 3s + lt — 15. 


11 . 2r + 3s — 4;= 12, 
3r — 2s =-l, 

8r — s — At — 10. 

12 . x + 2y — 2z = — 1, 
2x+ y 3- z= 5, 
— v + y — z = —2. 



13 . Use matrix inversion to détermine a production schedule that satisfies the 
requirements of the manufacturer described in Problem 12 of Section 2.1. 
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14. Use matrix inversion to détermine a production schedule that satisfies the 
requirements of the manufacturer described in Problem 13 of Section 2.1. 

15. Use matrix inversion to détermine a production schedule that satisfies the 
requirements of the manufacturer described in Problem 14 of Section 2.1. 

16. Use matrix inversion to détermine the bonus for the company described in 
Problem 16 of Section 2.1. 

17. Use matrix inversion to détermine the number of barrels of gasoline that the 
producer described in Problem 17 of Section 2.1 must manufacture to break 
even. 

18. Use matrix inversion to solve the Leontief input-output model described in 
Problem 22 of Section 2.1. 

19. Use matrix inversion to solve the Leontief input-output model described in 
Problem 23 of Section 2.1. 


3.4 Properties of the Inverse 

Theorem 1 If A, B, and C are square matrices of the same order with AB = I 
and CA = I, then B = C. 

Proof. C = CI = C(AB) = (CA)B = IB = B. □ 

Theorem 2 The inverse of a matrix is unique. 

Proof. Suppose that B and C are inverse of A. Then, by (1), we hâve that 

AB = I, BA = I, AC = I, and CA = I. □ 

It follows from Theorem 1 that B = C. Thus, if B and C are both inverses of A, 
they must in fact be equal. Hence, the inverse is unique. 

Using Theorem 2, we can prove some useful properties of the inverse of a 
matrix A when A is nonsingular. 

Property 1 (A -1 ) 1 = A. 

Proof. See Problem 1. □ 

Property 2 (AB) -1 = B -1 A -1 . 

Proof. (AB) -1 dénotés the inverse of AB. However, (B -1 A -1 ) (AB) = 
B -1 (A -1 A) B = B -1 IB = B -1 B = I. Thus, B -1 A -1 is also an inverse for AB, 
and, by uniqueness of the inverse, B -1 A -1 = (AB) - 1 . □ 


Property 3 (Ai A 2 • • • A„) 1 = A n 1 A n \ • • • A 2 1 A 1 1 . 
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Proof. This is an extension of Property 2 and, as such, is proved in a similar 
manner. □ 

Caution. Note that Property 3 States that the inverse of a product is not the 
product of the inverses but rather the product of the inverses commuted. 


Property 4 (A T ) 1 2 = (A 1 ) T 

Proof. (A t ) L dénotés the inverse of A T . However, using the property of the 
transpose that (AB) T = B T A T , we hâve that 

(A T )(A _1 ) T = (A -1 A) T = I T = I 


Thus, (A 1 ) T is an inverse of A T , and by uniqueness of the inverse, (A 1 ) 1 = 

(AT 1 . □ 


T 


Property 5 (AA) 1 = (1/A) (A) 1 if A is a nonzero scalar. 
Proof (AA) 1 dénotés the inverse of AA. However, 

(AA)(1/A)A -1 = A(1/A)AA _1 = II = I. 


Thus, (1/A) A 1 is an inverse of AA, and by uniqueness of the inverse (1/A) A 1 = 
(AA)" 1 . □ 


Property 6 The inverse of a nonsingular symmetric matrix is symmetric. 

Proof See Problem 18. □ 

Property 7 The inverse of a nonsingular upper or lower triangular matrix is 
again an upper or lower triangular matrix respectively. 

Proof This is immédiate from Theorem 2 and the constructive procedure 
described in Section 3.2 for calculating inverses. 

Finally, the inverse provides us with a straightforward way of defining square 
matrices raised to négative intégral powers. If A is nonsingular then we define 
A"" = (A" 1 )". □ 


Example 1 Find A 2 if 


1 1 

3 2 

1 

2 


A = 


1 
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Solution 



' 12 -6' 

2 

' 12 -6' 


' 12 -6' 


' 180 -96' 

-6 4 


-6 4_ 


-6 4_ 


-96 52 


Problems 3.4 


1. Prove Property 1. 

2 . Verify Property 2 for 




and B = 


'2 5' 

1 2 


3. Verify Property 2 for 



2 

4 


and 




4 . Verify Property 2 for 

1 2 -f 
0 1 - 1 . 
0 0 1 

5. Prove that (ABC) -1 = C -1 B -1 A -1 . 

6. Verify the resuit of Problem 5 if 



1 

1 

1 


A = 

0 

1 

1 

and B = 


0 

0 

1 





B = 


'4 0' 
0 2 


and 



-1 0 
2 2 • 


7. Verify Property 4 for the matrix A defined in Problem 2. 

8. Verify Property 4 for the matrix A defined in Problem 3. 

9. Verify Property 4 for the matrix A defined in Problem 4. 

10 . Verify Property 5 for X — 2 and 



1 0 
2 3 
-1 0 



11. Find A 2 and B 2 for the matrices defined in Problem 2. 
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12. Find A -3 and B -3 for the matrices defined in Problem 2. 

13. Find A -2 and B -4 for the matrices defined in Problem 3. 

14. Find A -2 and B -2 for the matrices defined in Problem 4. 

15. Find A -3 and B -3 for the matrices defined in Problem 4. 

16. Find A" 3 if 




17. If A is symmetric, prove the identity 


(BA“ 1 ) T (A“ 1 B t ) 1 = I. 

18. Prove Property 6. 


3.5 LU Décomposition 


Matrix inversion of elementary matrices (see Section 3.1) can be combined with 
the third elementary row operation (see Section 2.3) to generate a good numerical 
technique for solving simultaneous équations. It rests on being able to décomposé 
a nonsingular square matrix A into the product of lower triangular matrix L with 
an upper triangular matrix U. Generally, there are many such factorizations. If, 
however, we add the additional condition that ail diagonal éléments of L be unity, 
then the décomposition, when it exists, is unique, and we may write 


with 


L 


and 


U = 



A = 

LU 


" 1 

0 

0 ••• 

0 " 

h\ 

1 

0 ••• 

0 

h\ 

hi 

1 ••• 

0 

-Ini 

Ir2 

ln3 

1 _ 

Mil 

U\2 

U13 ••• 

Ml n 

0 

U 22 

M 23 • • • 

^2n 

0 

0 

U 33 

U 3 n 

0 

0 

0 ••• 

Mnn 



To décomposé A into from (7), we first reduce A to upper triangular from using 
just the third elementary row operation: namely, add to one row of a matrix a 
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scalar times another row of that same matrix. This is completely analogous to 
transforming a matrix to row-reduced form, except that we no longer use the 
first two elementary row operations. We do not interchange rows, and we do not 
multiply a row by a nonzero constant. Consequently, we no longer require the 
first nonzero element of each nonzero row to be unity, and if any of the pivots 
are zéro— which in the row-reduction scheme would require a row interchange 
operation— then the décomposition scheme we seek cannot be done. 

Example 1 Use the third elementary row operation to transform the matrix 



-1 3 
2 1 
-1 2 


into upper triangular form. 

Solution 



2 

-1 

3 " 


(N 
1 

-1 

3 " 

4 

2 

1 

— > 

0 

4 

-5 

-6 

-1 

2 


-6 

-1 

2 





F 2 

-1 

31 


0 4-5 

0 -4 11 


by adding to the 
second row (—2) times 
the first row 

, 

by adding to the 
third row ( 3 ) times 
the first row 


2 

0 

0 



by adding to the 

third row (1) times ■ 

the second row 


If a square matrix A can be reduced to upper triangular form U by a sequence 
of elementary row operations of the third type, then there exists a sequence of 
elementary matrices E21, E31, E41, . . . , E w>n _i such that 


(E n _i, n • • • E41E31E21) A = U, (8) 

where E21 dénotés the elementary matrix that places a zéro in the 2-1 position, 
E31 dénotés the elementary matrix that places a zéro in the 3-1 position, E41 
dénotés the elementary matrix that places a zéro in the 4-1 position, and so on. 
Since elementary matrices hâve inverses, we can write (8) as 

A=(E 2 - 1 1 E- 1 l E 4 -| l ...E fl -’_,)u. ( 9 ) 

Each elementary matrix in (8) is lower triangular. If follows from Property 7 of 
Section 3.4 that each of the inverses in ( 9 ) are lower triangular, and then from 
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Theorem 1 of Section 1.4 that the product of these lower triangular matrices is 
itself lower triangular. Setting 


L = e^e-Xi 1 • • ■: E”U 


we see that (9) is identical to (7), and we hâve the décomposition we seek. 


Example 2 Construct an LU décomposition for the matrix given in Example 1. 

Solution The elementary matrices associated with the elementary row opera- 
tions described in Example 1 are 


E 21 = 

“10 0 
-2 1 0 

, E 31 = 

1 

O H— ^ 

^ O 

O O 
1 

, and E 42 = 

1 

O H-* 
O 

O O 

1 


0 0 1 


3 0 1 


0 1 11 


with inverses given respectively by 


"1 

0 

0 


1 

0 

0" 


"1 

0 

0" 

2 

1 

0 

- E 3i 1 = 

0 

1 

0 

, and E 4 2 = 

0 

1 

0 

0 

0 

1 

-3 

0 

1 

0 

-1 

1 


Then, 


(N 

1 

-1 

3 " 


"1 

0 

1 

0 


1 

0 

1 

0 


"1 

0 

1 

0 


"2 

-1 

3 " 

4 

2 

1 

— 

2 

1 

0 


0 

1 

0 


0 

1 

0 


0 

4 

-5 

VO 

1 

1 

-1 

2 


0 

0 

1 


-3 

0 

1 


0 

-1 

1 


0 

0 

6 


or, upon multiplying together the inverses of the elementary matrices, 


" 2-13 


1 0 0 


"2 -1 3 " 

4 2 1 

— 

2 1 0 


0 4-5 

-6 -1 2 


-3 -1 1 


0 0 6 


Example 2 suggests an important simplification of the décomposition process. 
Note the éléments in L below the main diagonal are the négatives of the scalars 
used in the elementary row operations to reduce the original matrix to upper 
triangular form! This is no coincidence. In general, 

Observation 1 If an elementary row operation is used to put a zéro in the i— j 
position of A(z > j) by adding to row i a scalar k times row j, then the i— j element 
of L in the LU décomposition of A is — k. 

We summarize the décomposition process as follows: Use only the third ele- 
mentary row operation to transform a given square matrix A to upper triangular 
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from. If this is not possible, because of a zéro pivot, then stop; otherwise, the LU 
décomposition is found by defining the resulting upper triangular matrix as U and 
constructing the lower triangular matrix L utilizing Observation 1. 


Example 3 Construct an LU décomposition for the matrix 

"2 12 3" 

6 2 4 8 

A_ 1 -1 0 4 * 

0 1-3-4 


Solution Transforming A to upper triangular form, we get 


2 

1 

2 

3 


2 

1 

2 

3 

6 

2 

4 

8 


0 

-1 

-2 

-1 

1 

-1 

0 

4 


1 

-1 

0 

4 

0 

1 

-3 

-4 


0 

1 

-3 

-4 





"2 

1 

2 

3" 





0 

-1 

-2 

-1 





0 

3 

2 

-1 

5 

2 





0 

1 

-3 

-4 


"2 

1 

2 

3' 

0 

-1 

-2 

-1 

0 

0 

2 

4 

0 

1 

-3 

-4 


2 12 3 

0 -1 -2 -1 
0 0 2 4 

0 0-5-5 

2 12 3 

0 -1 -2 -1 
0 0 2 4 

0 0 0 5 


by adding to the 
second row (—3) times 
the first row 


by adding to the 
third row ( — \) times 
the first row 


by adding to the 
third row ( — |) times 
the second row 


by adding to the 
fourth row (1) times 
the second row 


by adding to the 
fourth row times 
the third row 


We now hâve an upper triangular matrix U. To get the lower triangular matrix 
L in the décomposition, we note that we used the scalar —3 to place a zéro in 
the 2-1 position, so its négative —(—3) = 3 goes into the 2-1 position of L. We 
used the scalar — \ to place a zéro in the 3-1 position in the second step of the 

above triangularization process, so its négative, becomes the 3-1 element in 
L; we used the scalar | to place a zéro in the 4-3 position during the last step of 
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the triangularization process, so its négative, — |, becomes the 4-3 element in L. 
Continuing in this manner, we generate the décomposition 


"2 

1 

2 

3" 


"1 

0 

0 

0“ 


"2 

1 

2 

3" 

6 

2 

4 

8 


3 

1 

0 

0 


0 

-1 

-2 

-1 

1 

-1 

0 

4 

— 

1 

2 

3 

2 

1 

0 


0 

0 

2 

4 

i — 

o 

1 

-3 

-4 


_0 

-1 

5 

2 

1_ 


0 

0 

0 

5 


LU décompositions, when they exist, can be used to solve Systems of simulta- 
neous linear équations. If a square matrix A can be factored into A = LU, then 
the System of équations Ax = b can be written as L(Ux) = b. To find x, we first 
solve the System 


Ly = b 



for y, and then, once y is determined, we solve the System 

Ux = y (11) 

for x. Both Systems (10) and (11) are easy to solve, the first by forward substitution 
and the second by backward substitution. 


Example 4 Solve the System of équations: 

2x — y + 3z = 9, 
4x + 2y + z = 9, 
—6x — y + 2z = 12. 

Solution This System has the matrix form 


r 2 

-1 3 


x 


' 9" 

'xf vo 
1 

1 

2 1 

-1 2 


i 

(X? 

1 

— 

9 

12 


The LU décomposition for the coefficient matrix A is given in Example 2. If 
we define the components of y by a, fi, and y, respectively, the matrix System 
Ly = b is 


1 

0 0 


a 


" 9" 

2 

-3 

1 0 

— 1 1 


P 

_y_ 

— 

9 

12 
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which is équivalent to the System of équations 

a — 9 , 

2a + P —9, 
—3a — P + y — 12. 


Solving this System from top to bottom, we get a — 9, — — 9, and y = 30. Conse- 

quently, the matrix System Ux = y is 


"2 -1 3 


X 


9" 

0 4-5 


T 

— 

-9 

0 0 6 


_z_ 


30 


which is équivalent to the System of équations 

2x — y + 3z — 9, 

4y — 5z — -9, 

6z — 30. 

Solving this System from bottom to top, we obtain the final solution x = — 1, y = 4, 
and z = 5. ■ 


Example 5 Solve the System: 

2 ci -\~ b T- 2c 3 d — 5, 

6rz + 2b + 4c + 8d — 8, 

a — b 4d — —4, 

b — 2c — 4d — —3. 


Solution The matrix représentation for this System has as its coefficient matrix 
the matrix A of Example 3. Define. 

y = [a, P, y, 5] T . 

Then, using the décomposition determined in Example 3, we can write the matrix 
System Ly = b as the System of équations 

a — 5, 

3a + P =8, 

y^a + jP + y — —4, 

— ^ — jy 4 - 8 — —3, 

which has as its solution a = 5, fi = —7, y — 4, and <5 = 0. Thus, the matrix System 
Ux = y is équivalent to the System of équations 

2ü T b T 2c T 3 d — 5, 

— b — 2c — d — —7, 

2c 4d — 4, 

5d = 0. 
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Solving this set from bottom to top, we calculate the final solution a — — 1, 
b — 3, c — 2, and d — 0. ■ 


LU décomposition and Gaussian élimination are equally efficient for solving 
Ax = b, when the décomposition exists. LU décomposition is superior when Ax = 
b must be solved repeatedly for different values of b but the same A, because once 
the factorization of A is determined it can be used with ail b. (See Problems 17 and 
18.) A disadvantage of LU décomposition is that it does not exist for ail nonsingular 
matrices, in particular whenever a pivot is zéro. Fortunately, this occurs rarely, and 
when it does the difficulty usually is overcome by simply rearranging the order of 
the équations. (See Problems 19 and 20.) 


Problems 3.5 

In Problems 1 through 14, A and b are given. Construct an LU décomposition for 
the matrix A and then use it to solve the System Ax = b for x. 


1. A = 


1 ï 
3 4 


b = 


1' 

-6 


2. A = 


2 ï 
1 2 


3. A = 

"8 3" 
5 2 

, b = 

"625" 

550 

4. A = 

1 1 0 
1 0 1 

, b = 

1 

1 






0 11 




b = 


' 11 ' 
-2 


5. A = 

"-1 

1 

2 

-3 

^ o 

i 

b = 

~-l“ 

-2 


2 

-2 

3 


3 



2 1 

3 



6. A = 

4 1 

0 

•> 

b = 


-2 -1 

-2 




"3 2 r 



"50" 

7. A = 

4 0 1 

ï 

b = 

80 


3 9 2 



20 


10 ' 

-40 

0 



1 

2 

-1 


80“ 

8. A = 

2 

0 

1 

b = 

159 


-1 

1 

3 


-75 


9. A = 

1 2 
0 2 

-1“ 

1 

b = 

1 

00 

1 

1 


o 

o 

i 

1 


1 

LT) 

1 



"1 

0 

i 

o 


i 

10. A = 

3 

2 

0 

b = 

4 


1 

1 

2 


2 
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11. A = 


1" 

1 

0 ’ 
1 


b = 


12. A = 


2 1 
1 4 
0 0 
0 1 


3" 

1 

1 ’ 
1 


4" 
-3 
-2 • 
-2 


1000 

200 

100 

100 


13. A = 


1" 

1 

2 ’ 

1 


b = 


14. A = 


2 0 
2 2 
-4 3 
1 0 


0" 

6 

1 ’ 
1 


-2 

4 

9 

4 


15. (a) Use LU décomposition to solve the System 

-x + 2 y = 9, 

2x + 3y = 4. 

(b) Résolve when the right sides of each équation are replaced by 1 and — 1, 
respectively. 

16. (a) Use LU décomposition to solve the System 

x + 3 y — z = —1, 

2x + 5y + z = 4, 

2x + 7y — 4z = —6. 


(b) Résolve when the right sides of each équation are replaced by 10, 10, and 
10, respectively. 

17. Solve the System Ax = b for the following vectors b when A is given as in 
Problem 4: 


5" 

(a) 7 , 

-4 


" 2 " 

(b) 2 , 
0 


"40" 

(c) 50 , 
20 


(d) 1 . 
3 
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18. Solve the System Ax = b for the following vectors b when A is given as in 
Problem 13: 



"-1 


"0 


"190 


T 

1 

(b) 

0 

> (c) 

130 

(d) 

1 

1 

0 

160 

1 

1 


0 


60 


1 


19. Show that LU décomposition cannot be used to solve the System 

2 y + z = -1, 

-r + y + 3z = 8, 

2 x — y — z = 1, 

but that the décomposition can be used if the first two équations are 
interchanged. 

20. Show that LU décomposition cannot be used to solve the System 

x + 2 y + z = 2, 

2x + 4y - z = 7, 

-r + y 4- 2z = 2, 

but that the décomposition can be used if the first and third équations are 
interchanged. 

21. (a) Show that the LU décomposition procedure given in this chapter cannot 

be applied to 



2 

9 


(b) Verify that A = LU, when 




and 



2 

7 


(c) Verify that A = LU, when 




and U = 


0 2 ' 
0 3 


(d) Why do you think the LU décomposition procedure fails for this A? What 
might explain the fact that A has more than one LU décomposition? 
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3.6 Final Comments on Chapter 3 

We now prove the answers to two questions raised earlier. First, what matrices 
hâve inverses? Second, if AB = I, is it necessarily true that AB = I too? 

Lemma 1 Let A and B ben x n matrices. If AB = I, then the System of équations 
Ax = y has a solution for every choice ofthe vector y. 

Proof. Once y is specified, set x = By. Then 


Ax = A(By) = (AB)y = Iy = y, 


SO X = By is a solution of Ax = y. □ 

Lemma 2 If A and B are n x n matrices with AB = I, then A has rank n. 

Proof. Designate the rows of A by Ai , A 2 , . . . , A n . We want to show that these n 
rows constitute a linearly independent set of vectors, in which case the rank of A is 
n. Designate the columns of I as the vectors ei, e 2 , . . . , e n , respectively. It follows 
from Lemma 1 that the set of équations Ax = e 7 - (j = 1, 2, . . . , n) has a solution 
for each j. Dénoté these solutions by xi, X 2 , . . . x„ , respectively. Therefore, 

Ax j — ej. 

Sincee ; - (j — 1,2 , ,n) is ann-dimensionalcolumn vector having aunity element 
in row j and zéros everywhere else, it follows form the last équation that 

1 when i — j, 

0 when i 7 ^ j. 

This équation can be notationally simplified if we make use of the Kronecker delta 
8ij defined by 

1 when i — j. 

0 when i j. 

Then, 



A/ Xj = 


A/'Xy — 8ij. 


c i^i = 0 . 


1=0 


Now consider the équation 
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We wish to show that each constant a must be zéro. Multiplying both sides of this 
last équation on the right by the vector x ; -, we hâve 


(fl f ' A 'j x j - 0x /’ 

n 

Y (ciAi) xj = 0, 
i = o 

n 

^c/(A z x y ) = 0, 
i = o 

n 

Y; C i^ij — 0 , 

1=0 

cj = 0. 

Thus for each xj (j = 1, 2, . . . , n) we hâve cj — 0, which implies that c\—C 2 — 
• . . = c n = 0 and that the rows Ai, A 2 , . . . , A n are linearly independent. □ 

It follows directly from Lemma 2 and the définition of an inverse that if an 
n x n matrix A has an inverse, then A must hâve rank n. This in turn implies 
directly that if A does not hâve rank n , then it does not hâve an inverse. We now 
want to show the converse: that is, if A has rank n , then A has an inverse. 

We already hâve part of the resuit. If an n x n matrix A has rank n , then the 
procedure described in Section 3.2 is a constructive method for obtaining a matrix 
C having the property that CA = I. The procedure transforms A to an identity 
matrix by a sequence of elementary row operations Ei, E 2 , . . . , E^_i, E*. That is, 

E^E^—i . . . E 2 E 1 A = I. 

Setting 

C = EkEk-i . . . E 2 E 1 , (12) 


we hâve 


CA = I. 



We need only show that AC = I, too. 

Théo rem 1 If A and B are n x n matrices such that AB = I, then B A = I. 

Proof. If AB = I, then from Lemma 1 A has rank n , and from (12) and (13) 
there exists a matrix C such that CA = I. It follows fromTheorem 1 of Section 3.4 
that B = C. □ 
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The major implication of Theorem 1 is that if B is a right inverse of A, then B 
is also a left inverse of A; and also if A is a left inverse of B, then A is also a right 
inverse of B. Thus, one needs only check whether a matrix is a right or left inverse; 
once one is verified for square matrices, the other is guaranteed. In particular, if 
annxfl matrix A has rank n , then (13) is valid. Thus, C is a left inverse of A. As a 
resuit of Theorem 1, however, C is also a right inverse of A— just replace A with 
C and B with A in Theorem 1— so C is both a left and right inverse of A, which 
means that C is the inverse of A. We hâve now proven: 

Theorem 2 An n x n matrix A has an inverse if and only if A has rank n. 




An Introduction to 
Optimization 


4.1 Graphing Inequalities 

Many times in real life, solving simple équations can give us solutions to everyday 
problems. 


Example 1 Suppose we enter a supermarket and are informed that a certain 
brand of coffee is sold in 3-lb bags for $6.81. If we wanted to détermine the cost 
per unit pound, we could model this problem as follows: 

Let x be the cost per unit pound of coffee; then the following équation represents 
the total cost of the coffee: 


x-\-x-\-x — 3x — 6.81. 



Dividing both sides of (1) by 3 gives the cost of $2.27 per pound of coffee. ■ 


Example 2 Let’s suppose that we are going to rent a car. If the daily fixed cost 
is $100.00, with the added price of $1.25 per mile driven, then 

C = 100+ 1.25m (2) 

represents the total daily cost, C, where m is the number of miles traveled on a 
particular day. 

What if we had a daily budget of $1000.00? We would then use (2) to déter- 
mine the number of miles we could travel given this budget. Using elementary 
algebra, we see that we would be able to drive 720 miles. ■ 

These two simple examples illustrate how équations can assist us in our daily 
lives. But sometimes things can be a bit more complicated. 
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Example 3 Suppose we are employed in a factory that produces two types of 
bicycles: a standard model (S) and a deluxe model ( D ). Let us assume that the 
revenue (R) on the former is $250 per bicycle and the revenue on the latter is $300 
per bicycle. Then the total revenue can be expressed by the following équation: 

R = 2505 + 300 Z). (3) 

Now suppose manufacturing costs are $10,000; so to make a profit, R has to be 
greater than $10,000. Hence the following inequality is used to relate the bicycles 
and revenue with respect to showing a profit: 

2505 + 300 Z) > 10,000. (4) 

Relationship (4) illustrâtes the occurrence of inequalities. However, before we can 
solve problems related to this example, it is important to “visualize” inequalities, 
because the graphing of such relationships will assist us in many ways. ■ 

For the rest of this section, we will sketch inequalities in two dimensions. 


Example 4 Sketch the inequality x + y <2. The équation x + y = 2 is a str- 
aight line passing through the points (2,0)— the x-intercept— and (0,2)— the 
y-intercept. The inequality x + y < 2 merely includes the région “under” the 
line. 



Figure 4.1 
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Remark 1 Notice that the lower left-hand part of the graph is shaded. 
An easy way to check is to pick a point, say (—50, —50); clearly —50 + 
-50 < 2, therefore the “half-region” containing this point must be the shaded 
portion. 

Remark 2 The graph of the strict inequality x + y < 2 yields the same picture 
with the line dashed (instead of solid) to indicate that points on the line x + y = 2 
are not included. 


Example 5 Sketch 2x + 3y > 450. 


1000 


800 


600 


400 


200 



Figure 4.2 



Remark 3 Notice that we hâve restricted this graph to the first quadrant. 
Many times the variables involved will hâve non-negative values, such as vol- 
ume, area, etc. Notice, too, that the région is infinité , as is the région in 
Example 4. 


Example 6 Sketch 4x + y < 12 and 2x + 5y < 24, where x > 0 and y > 0. 
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Figure 4.3 


Remark 4 Note that the “upper-right” corner point is (2,4). This point is 
the intersection of the straight lines given by the équations 4x + y = 12 and 
2x -b 5 y = 24; in Chapter 2 we covered techniques used in solving simultaneous 
équations. Here the added constraints of x > 0 and y > 0 render a bounded or 
finite région. 

We will see régions like Figure 4.3 again both in Section 4.2 (with regard to 
modeling) and Section 4.3 (using the technique of linear programming). 

Problems 4.1 

Sketch the following inequalities: 

1. y < 0 

2. x >0 

3. y > 7i 

4. x + 4y < 12 

5. x + 4y < 12 

6. x + 4y > 12 

7. x + 4y > 12 
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Sketch the inequalities on the same set of axes: 

8 . x + 4y < 12, x > 0, y > 0 

9. x + 4y < 12, 5x + 2y < 24 

10 . x + 4y > 12, 5x + 2y > 24 

11. x + 2y < 12, 2x + y < 16, x + 2 y < 20 

12. x — y > 100 

13 . x + y > 100, 3x + 3 y < 60 

14 . x + J < 10, — x + y < 10, x — y < 10, — x — y < 10 

4.2 Modeling with Inequalities 

Consider the following situation. Suppose a toy company makes two types of 
wagons, X and Y. Let us further assume that during any work period, each X takes 
3 hours to construct and 2 hours to paint, while each Y takes 1 hour to construct and 
2 hours to paint. Finally, the maximum number of hours allotted for construction 
is 1500 and the limit on hours available for painting is 1200 hours. If the profit on 
each X is $50 and the profit on each Y is $60, how many of each type of wagon 
should be produced to maximize the profit? 

We can model the above with a number of inequalities. First, we must define 
our variables. Let X represent the number of X wagons produced and Y represent 
the number of Y wagons produced. This leads to the following four relationships: 


3 X + Y < 1500 

( 5 ) 

2X + 2 Y < 1200 

(6) 

X>0 

( 7 ) 

Y > 0. 

(8) 


Note that (5) represents the constraint due to construction (in hours) while (6) 
represents the constraint due to painting (also in hours). The inequalities (7) and 
(8) merely State that the number of each type of wagon cannot be négative. 

These four inequalities can be graphed as follows in Figure 4.4: 

Let us make a few observations. We will call the shaded région that satisfies 
ail four inequalities the région of feasibility. Next, the shaded région has four 
“corner points” called vertices. The coordinates of these points are given by (0, 0), 
(0, 600), (450, 150) and (500, 0). Lastly, this région has the property that, given 
any two points in the interior of the région, the straight line segment connecting 
these two points lies entirely within the région. We call régions with this property 


convex. 
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Figure 4.4 


The folio wing équation gives the profit (in dollars): 

P(X, Y) = 50X + 60 Y. (9) 

Note that Equation (9) is called the objective function. The notation P(X, Y) is read 
“P of X and T” and is evaluated by simply substituting the respective values into 
the expression. For example, P (0,600) = 50(0) + 60(600) = 0 + 36,000 = 36,000 
dollars, while P (450, 150) = 50(450) + 60(150) = 22,500 + 9000 = 31,500 dollars. 

Equation (9), the inequalities (5)-(8), and Figure 4.4 model the situation 
above, which is an example of an optimization problem. In this particular exam- 
ple, our goal was to maximize a quantity (profit). Our next example deals with 
minimization. 

Suppose a spécifie diet calls for the following minimum daily requirements: 186 
units of Vitamin A and 120 units of Vitamin B. Pill X contains 6 units of Vitamin 
A and 3 units of Vitamin B, while pill Y contains 2 units of Vitamin A and 2 units 
of Vitamin B. What is the least number of pills needed to satisfy both vitamin 
requirements? 

Let us allow X to represent the number of X pills ingested and let Y represent 
the number of Y pills taken. Then the following inequalities hold: 


6X + 2Y > 186 
3X + 2 Y > 120 


( 10 ) 

( 11 ) 
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Figure 4.5 


X>0 (12) 

Y > 0. (13) 

Note that (10) models the minimum daily requirement of units of Vitamin A, while 
(11) refers to the minimum daily requirement of units of Vitamin B. The quantity 
to be minimized, the total number of pills, is given by the objective function: 


N(X, Y) = X + Y. (14) 

We note that while this région of feasibility is convex, it is also unbounded. Our 
vertices are (40, 0), (0, 93), and (22, 27). 

In the next section we will solve problems such as these by applying a very 
simple, yet extremely powerful, theorem of linear programming. 


Problems 4.2 

Model the following situations by defining ail variables and giving ail inequalities, 
the objective function and the région of feasibility. 

1. Farmer John gets $5000 for every truck of wheat sold and $6000 for every truck 
of corn sold. He has two fields: field A has 23 acres and field B has 17 acres. 
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For every 2 acres of field A, Farmer John produces a truck of wheat, while 
3 acres are required of field B for the same amount of wheat. Regarding the 
corn, 3 acres of field A are required for a truck, while only 1 acre of field B is 
needed. How many trucks of each commodity should be produced to maximize 
Farmer John’s profit? 

2. Redo Problem (1) if Farmer John gets $8000 for every truck of wheat and $5000 
for every truck of corn. 

3. Dr. Lori Pesciotta, a research scientist, is experimenting with two forms of a 
spécial compound, H-Turebab. She needs at least 180 units of one form of the 
compound (a) and at least 240 units of the second form of the compound (fi). 
Two mixtures are used: X and Y. Every unit of X contains two units of a and 
three units of /3, while each unit of Y has the opposite concentration. What 
combination of X and Y will minimize Dr. Pesciotta’s costs, if each unit of X 
costs $500 and each unit of Y costs $750? 

4. Redo Problem (3) if X costs $750 per unit and Y costs $500 per unit. 

5. Redo Problem (3) if, in addition, Dr. Pesciotta needs at least 210 units of a 
third form (y) of H-Turebab , and it is known that every unit of both X and Y 
contains 10 units of y. 

6. Cereal X costs $.05 per ounce while Cereal Y costs $.04 per ounce. Every 
ounce of X contains 2 milligrams (mg) of Zinc and 1 mg of Calcium, while 
every ounce of Y contains 1 mg of Zinc and 4 mg of Calcium. The minimum 
daily requirement (MDR) is 10 mg of Zinc and 15 mg of Calcium. Find the 
least expensive combination of the cereals which would satisfy the MDR. 

7. Redo Problem (6) with the added constraint of at least 12 mg of Sodium if each 
ounce of X contains 3 mg of Sodium and every ounce of Y has 2 mg of Sodium. 

8. Redo Problem (7) if Cereal X costs $.07 an ounce and Cereal Y costs $.08 an 
ounce. 

9. Consider the following group of inequalities along with a corresponding objec- 
tive function. For each one, sketch the région of feasibility (except for 9 g) and 
construct a scénario that might model each set of inequalities: 

(a) v > 0, y > 0, 2x + 5y < 10, 3x + 4y < 12, F(x, y) = lOOx + 55y 

(b) v > 0, y < 0, x + y < 40, x + 2y < 60, G(x, y) = Ix + 6y 

(c) v > 2, y > 3, v + y < 40, x + 2y < 60, //(x, y) = x + 3y 

(d) v > 0, y > 0, x + y < 600, 3x + y < 900, x + 2y < 1000, J(x , y) = 10* + 4y 

(e) 2x + 9y > 1800, 3x + y > 750, K(x, y) = 4x + lly 

(f ) x + y > 100, x + 3y > 270, 3x + y > 240, L(x, y) = 600x + 375y 

(g) x > 0, y > 0, z > 0, x + y + 2z < 12, 2x + y + z < 14, x + 3y + z < 15, 

M(x, y, z) = 2x + 3y + 4z (Do not sketch the région of feasibility for this 
problem.) 
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4.3 Solving Problems Using Linear Programming 

We are now ready to solve a fairly large class of optimization problems using 
a spécial form of the Fundamental Theorem of Linear Programming. We will 
not prove this theorem, but many references to the proof of a more general 
resuit are available (for example, see Luenberger, D. G., Linear and Nonlinear 
Programming , 2nd Ed., Springer 2003). 

The Fundamental Theorem of Linear Programming Let r be a convex région 
of feasibility in the xy-plane. Then the objective function F(x, y) = ax + by, where 
a and b are real numbers, takes on both maximum and minimum values— if they 
exist— on one or more vertices of T. 

Remark 1 The theorem holds only if maximum and/or minimum values exist. 

Remark 2 It is possible to hâve infinitely many values where an optimal (max- 
imum or minimum) value exists. In this case, they would lie on one of the line 
segments that form the boundary of the région of feasibility. See Example 3 below. 

Remark 3 The word programming has nothing to do with computer program- 
ming, but rather the systematic order followed by the procedure, which can also 
be termed an algorithm. 

Some examples are in order. 


Example 1 (Wagons): Consider the inequalities (5) through (8), along with 
Equation (9), from Section 4.2. We again give the région of feasibility below 
in Figure 4.6 (same as Figure 4.4): 

Evaluation our objective function, 

P(X, Y) = 50X + 60F, (15) 

at each of the four vertices yields the following results: 


>( 0 , 0 ) =0 
P (0,600) = 36,000 
P (450, 150) = 31,500 
P (500,0) = 25,000. 


By the Fundamental Theorem of Linear Programming, we see that the maxi- 
mum profit of $36,000 occurs if no X wagons are produced and 600 Y wagons are 
made. ■ 
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Figure 4.6 


Example 2 (Wagons): Suppose the profit function in the previous example is 
given by 

#(X, Y) = SOX + 50 K (16) 

Then 


#( 0 , 0 ) = 0 
#(0,600) = 30,000 
#(450,150) = 43,500 
#(500,0) = 40,000. 


We see, in this situation, that the maximum profit of $43,500 occurs if 450 X 
wagons are produced, along with 150 Y wagons. ■ 


Example 3 (Wagons): Consider the examples above with the profit function 
given by 


L(X, Y) = 75X + 75 Y 


(17) 
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Then 


L( 0,0) = 0 
L( 0,600) = 45,000 
L(450, 150) = 45,000 
L (500,0) = 37,500. 

Note that we hâve two situations in which the profit is maximized at $45,000; in 
fact, there are many points where this occurs. For example, 

L(300, 300) = 45,000. (18) 

This occurs at any point along the constraint given by inequality (2). The reason 
lies in the fact that coefficients of X and Y in (2) and in Equation (7) hâve the same 
ratio. ■ 


Example 4 (Vitamins): Consider constraints (10) through (13) above in 
Section 4.2; minimize the objective function given by Equation (14). 


N(X, Y) = X + Y. 



The région of feasibility (same as Figure 4.5) is given below in Figure 4.7: 
Evaluating our objective function (19) at the three vertices, we find that 

iV(40, 0) = 40 
N(0, 93) = 93 
N (22, 27) = 49, 

, 

so the minimum number of pills needed to satisfy the minimum daily requirement 
is 40. 

Sometimes a constraint is redundant ; that is, the other constraints “include” 
the redundant constraint. 

For example, suppose we want to maximize the objective function 


given the constraints 


Z(X, Y) = 4X + 3 T, 

(20) 

4X + 2Y < 40 

(21) 

3X + 4Y < 60 

(22) 

X>0 

(23) 

Y > 0. 

(24) 
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Figure 4.7 


The vertices of the région of feasibility are (0, 0), (0, 15), (4, 12), and (10, 0), as 
seen below in Figure 4.8. 

Note that (11) is maximized at Z( 4, 12) = 52. 

Suppose we now add a third constraint: 

Z + Y < 30. (25) 

Figure 4.9 below reflects this added condition. Note, however, that the région of 
feasibility is not changed and the four vertices are unaffected by this redundant 
constraint. It folio ws, therefore, that our objective function Z(X, Y) = 4X + 3F is 
still maximized at Z(4, 12) = 52. 

Remark 4 Sometimes a vertex does not hâve whole number coordinates (see 
problem (15) below). If the physical model does not make sense to hâve a frac- 
tional or décimal answer— for example 2.5 bicycles or 1/3 cars— then we should 
check the closest points with whole number coordinates, provided these points 
lie in the région of feasibility. For example, if (2.3, 7.8) is the vertex which gives 
the optimal value for an objective function, then the following points should be 
checked: (2, 7), (2, 8), (3, 7) and (3, 8). 




• • 
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Problems 4.3 

Using linear programming techniques, solve the folio wing problems. 

1. Section 4.2, Problem (1). 

2. Section 4.2, Problem (2). 

3. Section 4.2, Problem (3). 

4. Section 4.2, Problem (4). 

5. Section 4.2, Problem (5). 

6. Section 4.2, Problem (6). 

7. Section 4.2, Problem (7). 

8 . Section 4.2, Problem (8). 

9. Section 4.2, Problem (9a); maximize F(x, y). 

10. Section 4.2, Problem (9b); maximize G(x, y). 

11. Section 4.2, Problem (9c); maximize H(x , y). 

12. Section 4.2, Problem (9d); maximize J(x, y). 

13. Section 4.2, Problem (9e); minimize K(x, y). 

14. Section 4.2, Problem (9f); minimize L(x, y). 

15. Maximize P(x, y) = 7x + 6y subject to the constraints x > 0, y > 0, 2x + 3y < 
1200 and 6x + y < 1500. 


4.4 An Introduction to the Simplex Method 

In most of the problems considered in the previous section, we had but two 
variables (usually X and F) and two constraints, not counting the usual condi- 
tions of the non-negativity of X and Y. Once a third constraint is imposed, the 
région of feasibility becomes more complicated; and, with a fourth constraint, 
even more so. 

Also, if a third variable, say Z, is brought into the discussion, then the 
région of feasibility becomes three-dimensionall This certainly makes the tech- 
nique employed in the previous section much more difficult to apply, although 
theoretically it can be used. 

We are fortunate that an alternate method exists which is valid for any number 
of variables and any number of constraints. It is known as the Simplex Method. 
This is a classic method that has been in use for many years. The reader may wish 
to consult G. Hadley’s Linear Programming published by Addison-Wesley in 1963 
for the theoretical underpinnings of this algorithm. 

Before we illustrate this technique with a number of examples, describing and 
defining terms as we go along, we point out that this section will deal exclusively 
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with maximization problems. We will address minimization in the next, and final, 
section of this chapter. 


Example 1 Suppose we want to maximize the following function of two 
variables: 


z = 7xi + 22x2. (26) 

Note that we are using x; instead of the usual x and y, due to the fact that, in later 
examples, we will hâve more than two independent variables. 

Let us assume that the following constraints are imposed: 


3xi + 10x2 < 33,000 

(27) 

5xi + 8x2 < 42,000 

(28) 

xi > 0 

(29) 

X2 > 0 . 

(30) 


We now introduce the concept of slack variables, which we dénoté by 57. These 
variables (which can never be négative) will “pick up the slack” in the relationships 
(27) and (28) and convert these inequalities into équations. That is, (27) and (28) 
can now be written respectively as: 

3xi + 10x2 + s\ = 33,000 (31) 


and 


5xi + 8x2 + S 2 — 42,000. (32) 

We also incorporate these slack variables into our objective function (26), rewriting 
it as: 


—7xi — 22x2 + Osi + 0^2 + z = 0. (33) 

Finally, we re write (27) and (28) as 

3xi + 10x2 + $1 + 0^2 + 0z = 33,000 (34) 

5xi + 8x2 + Osi + 1$2 + 0z = 42,000. (35) 


Remark 1 Admittedly, the Equations (33) through (35) seem somewhat strange. 
However, the reader will soon see why we hâve recast these équations as they now 
appear. 
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We are now ready to put these last three équations into a table known as the 
initial tableau. This is nothing more than a kind of augmented matrix. To do this, 
we merely “detach” the coefficients of the five unknowns (jq, jq, q, S 2 , and z) and 
form the following table: 


XI 

X2 

■si 

^2 

Z 


3 

10 

1 

0 

0 

33,000 

5 

8 

0 

1 

0 

42,000 

-7 

-22 

0 

0 

1 

0 



Remark 2 Note that the objective function équation— here, Equation (33) — 
is in the bottom row. Also, unless otherwise stipulated, we shall always assume 
that the decision variables— that is, x\ and jq — are non-negative. Notice, too, the 
vertical bar that appears to the left of the rightmost column and the horizontal bar 
placed above the bottom row. Finally, we point out that the entry in the last row 
and last column is always zéro for this initial tableau. These conventions will assist 
us in interpreting the end State of the Simplex Method. 

Before continuing with the Simplex Method, let us consider another example. 


Example 2 Put the following maximization problem into the initial tableau 
form: z = Ax\ + 7x2 + 9 * 3 , where x\ + X 2 + 6 x 3 < 50, 2xi + 3 x 2 < 40, and 4xi + 
9x2 + 3x3 <10. 

Note that we hâve three independent (decision) variables (the x z ) and that the 
three constraints will give us three slack variables (the 57). These lead us to the 
following four équations: 


—4xi — 7x2 — 9x3 + 0 s\ + 0^2 + O53 + z = 0 (37) 

xi + X2 + 6x3 + + 0^2 + 0^3 + 0z = 50 (38) 

2xi + 0x2 + 3x3 + Oq + S2 + 0^3 -h 0z = 40 (39) 

4xi + 9x2 + 3x3 + 0q + 0^2 + + 0z = 10. (40) 

The initial tableau for this example is given below: 


xi 

x 2 

*3 

■Si 

^2 

^3 

Z 


1 

1 

6 

1 

0 

0 

0 

50 

2 

0 

3 

0 

1 

0 

0 

40 

4 

9 

3 

0 

0 

1 

0 

10 

-4 

-7 

-9 

0 

0 

0 

1 

0 


( 41 ) 
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We will now outline the steps in the Simplex Method: 

• Change ail inequalities into équations via the use of slack variables. 

• Rewrite the objective function, z, in terms of slack variables, setting one side 
of the équation equal to zéro and keeping the coefficient of z equal to +1. 

• The number of équations should equal the sum of the constraints plus one (the 
équation given by the objective function). 

• Form the initial tableau, listing the constraints above the objective function, 
labeling the columns, beginning with the decision variables, followed by the 
slack variables, with z represented by the last column before the vertical bar. 
The last column should hâve ail the “constants.” 

• Locate the most négative number in the last row. If more than one equally nég- 
ative number is présent, arbitrarily choose any one of them. Call this number 
k. This column will be called the work column. 

• Consider each positive element in the work column. Divide each of these 
éléments into the corresponding row entry element in the last column. The 
ratio that is the smallest will be used as the work column’s pivot. If there is 
more than one smallest ratio, arbitrarily choose any one of them. 

• Use elementary row operations (see Chapter 2) to change the pivot element 
to 1, unless it is already 1. 

• Use elementary row operations to transform ail the other éléments in the work 
column to 0. 

• A column is reduced when ail the éléments are 0, with the exception of the 
pivot, which is 1. 

• Repeat the process until there are no négative éléments in the last row. 

• We are then able to détermine the answers from this final tableau. 

Let us illustrate this by returning to Example 1, where the initial tableau is 
given by 


XI 

X2 

si 

s 2 

Z 


3 

10 

1 

0 

0 

33,000 

5 

8 

0 

1 

0 

42,000 

-7 

-22 

0 

0 

1 

0 



We first note that —22 is the most négative number in the last row of (42). So the 
“*2” column is our work column. 

We next divide 33,000 by 10 = 3300 and 42,000 by 8 = 5250; since 3300 is the 
lesser positive number , we will use 10 as the pivot. Note that we hâve put a carat 
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(a) over the 10 to signify it is the pivot element. 


XI 

X2 

si 

s 2 

Z 



3 

/v 

10 

1 

0 

0 

33,000 ' 

(43) 

• 

5 

8 

0 

1 

0 

42,000 

-7 

-22 

0 

0 

1 

0 



We now divide every element in the row containing the pivot by 10. 


XI 

X2 

■si 

s 2 

Z 



0.3 

/v 

1 

0.1 

0 

0 

3300 

(44) 

• 

5 

8 

0 

1 

0 

42,000 

-7 

-22 

0 

0 

1 

0 



Next, we use elementary row operations; we multiply the first row by — 8 and add 
it to the second row and multiply the first row by 22 and add it to the third row. 
This will give us a 0 for every element (other than the pivot) in the work column. 


XI 

xi 

■si 

s 2 

Z 



0.3 

/V 

1 

0.1 

0 

0 

3300 

(45) 

• 

2.6 

0 

-0.8 

1 

0 

15,600 

-0.4 

0 

2.2 

0 

1 

72,600 



And now we repeat the process because we still hâve a négative entry in the 
last row; that is, —0.4 is in the “xi” column. Hence, this becomes our new work 
column. 

Dividing 3300 by 0.3 yields 11,000; dividing 15,600 by 2.6 gives us 6000; since 
6000 is the lesser of the two positive ratios, we will use the 2.6 entry as the pivot 
(again denoting it with a carat, and removing the carat from our first pivot). 


X\ 

0.3 

X2 

1 

■si 

0.1 

s 2 

0 

Z 

0 

3300 " 

(46) 

• 

/V 

2.6 

0 

-0.8 

1 

0 

15,600 

-0.4 

0 

2.2 

0 

1 

72,600 _ 



Dividing each element in this row by 2.6 gives us the following tableau: 


X\ 

X2 

■si 

s 2 

z 



0.3 

/V 

1 

0.1 

0 

0 

3300 ' 

(47) 

1 

0 

-.31 

.38 

0 

6000 

\ / 

• 

-0.4 

0 

2.2 

0 

1 

72,600 



Using our pivot and elementary row operations, we transform every other element 
in this work column to 0. That is, we multiply each element in the second row by 
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—0.3 and add the row to the first row and we multiply every element in the second 
row by 0.4 and add the row to the last row. This gives us the following tableau: 


XI 

*2 

si 

s 2 

z 


0 

1 

0.19 

-0.12 

0 

1500 

1 

0 

-.31 

.38 

0 

6000 

0 

0 

2.08 

0.15 

1 

75,000 



We are now finished with the process, because there are no négative éléments in 
the last row. We interpret this final tableau as follows: 

• x\ — 6000 (note the “1” in the x\ column and the “0” in the X 2 column). 

• X 2 = 1500 (note the “0” in the x\ column and the “1” in the X 2 column). 

• Both slack variables equal 0. To verify this, please see Equations (31) and (32) 
and substitute our values for x\ and X 2 into these équations. 

• The maximum value of z is 75,000 (found in the lower right-hand corner box). 
We now give another example. 


Example 3 Maximize z = x\ + 2x2, subject to the constraints 4jq + 2x2 < 40 
and 3xi + 4x2 < 60. 

Following the practice discussed in this section and introducing the slack 
variables, we hâve: 


4xi + 2x2 + = 40 (49) 

3xi + 4x2 + s 2 — 60 (50) 


and 


—xi — 2x2 + z = 0. 



We form the initial tableau, using coefficients of 0 where needed, as follows: 


xi 

X2 

■si 

s 2 

Z 


4 

2 

1 

0 

0 

40 

3 

4 

0 

1 

0 

60 

-1 

-2 

0 

0 

1 

0 



The second column will be our work column, since —2 is the most négative entry. 
Dividing 40 by 2 gives 20; dividing 60 by 4 yields 15. Since 15 is a lesser positive 
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ratio than 20, we will use the 4 as the pivot: 


Xî 

X2 

■si 

S2 

Z 



4 

2 

/V 

1 

0 

0 

40 ” 

(53) 

3 

4 

0 

1 

0 

60 

• 

-1 

-2 

0 

0 

1 

0 _ 



Dividing every element of the second row will make our pivoting element 1: 


XI 

X2 

■si 

^2 

z 



4 

2 

/V 

1 

0 

0 

40 ' 

(54) 

0.75 

1 

0 

0.25 

0 

15 

• 

-1 

-2 

0 

0 

1 

0 _ 



We now use our pivot, along with the proper elementary row operations, to make 
every other element in the column zéro. This leads to the following tableau: 


xi 

XI 

■Si 

«2 

Z 



2.5 

0 

/V 

1 

-0.5 

0 

10 " 

(55) 

0.75 

1 

0 

0.25 

0 

15 

• 

0.5 

0 

0 

0.5 

1 

30 _ 



Since the last row has no négative entries, we are finished and hâve the final 
tableau: 


x\ 

X2 

■si 

^2 

z 



' 2.5 

0 

1 

-0.5 

0 

10 " 

(56) 

• 

0.75 

1 

0 

0.25 

0 

15 

0.5 

0 

0 

0.5 

1 

30 



This final tableau is a little more complicated to interpret than (48). 

First notice the “1” in the second row; this implies that X 2 = 15. The 
corresponding équation represented by this second row thereby reduces to 

0.75jq + 15 + 0.25*2 = 15. (57) 

Which forces both x\ and *2 to be zéro, since neither can be négative. This forces 
*1 = 10, as we can infer from the équation represented by the first row: 


0.25xi + — 0.5*2 = 10- (58) 

In practice, we are not concerned with the values of the slack variables, so we sum- 
marize by simply saying that our answers are x\ — 0 and %2 — 15 with a maximum 
value of z = 30. ■ 
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As we hâve pointed out, this is a classic technique. However, as the number 
of variables (decision and/or slack) increases, the calculations can be somewhat 
burdensome. Thankfully, there are many software packages to assist in this matter. 
Please refer to the Final Comments at the end of this chapter. 

One final remark: As is the case with linear programming, if there are an infinité 
number of optimal solutions, the Simplex Method does not give ail solutions. 


Problems 4.4 

Using the Simplex Method, solve the following problems: 

1. Section 4.2, Problem (1). 

2. Section 4.2, Problem (2). 

3. Maximize z = 3jq + 5x2, subject to x\ + *2 < 6 and 2jq + X 2 < 8. 

4. Maximize z = 8jq 4- X2, subject to the same constraints in (3). 

5. Maximize z = x\ + 12x2, subject to the same constraints in (3). 

6. Maximize z = 3xi 4- 6x2, subject to the constraints x\ 4- 3x2 < 30, 2xi 4- 2x2 < 

40, and 3xi 4- X2 < 30. 

7. Consider problem (9) at the end of Section 4.2. Set up the initial tableaus for 
problems (9a) through (9d). 


4.5 Final Comments on Chapter 4 

In this chapter we covered two approaches to optimization, the Linear Program- 
ming Method and the Simplex Method. Both of these techniques are classical 
and their géométries and algebraic simplicity reflect both the beauty and power 
of mathematics. 

Our goal was to introduce the reader to the basics of these “simple” methods. 
However, he or she should be cautioned with regard to the underlying theory. 
That is, many times in mathematics we hâve élégant results (theorems) which 
are proved using very deep and subtle mathematical concepts with respect to the 
proofs of these theorems. 

As we mentioned in the last section, the calculations, while not difficult, can 
be a burden. Calculators and software packages can be of great assistance here. 

We close with two observations. Please note that we hâve considered very 
spécial cases where the constraints of the “<” variety had positive quantifies 
on the right-hand side. If this is not the case for ail the constraints, then we 
must use an enhanced version of the Simplex Method (see, for example, Finite 
Mathematics : A Modeling Approach by R. Bronson and G. Bronson published 
by West in 1996). 
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Similarly, regarding the solving of minimization problems via the Simplex 
Method, we essentially consider the “négation” of the objective function, and 
then apply a modified version of the Simplex Method. For example, suppose we 
wanted to minimize z — 3x\ + 2x2, subject to the same constraints. In this case we 
would maximize Z — —z — —2x\ — 2x2 while recasting our constraints, and then 
proceed with the Simplex Method. 



Déterminants 



5.1 Introduction 


Every square matrix has associated with it a scalar called its déterminant. To be 
extremely rigorous we would hâve to define this scalar in terms of permutations 
on positive integers. However, since in practice it is difficult to apply a définition 
of this sort, other procedures hâve been developed which yield the déterminant in 
a more straightforward manner. In this chapter, therefore, we concern ourselves 
solely with those methods that can be applied easily. We note here for reference 
that déterminants are only defined for square matrices. 

Given a square matrix A, we use det(A) or | A| to designate its déterminant. If 
the matrix can actually be exhibited, we then designate the déterminant of A by 
replacing the brackets by vertical straight fines. For example, if 



2 3 
5 6 
8 9 



then 


det(A) = 


1 

4 

7 


2 

5 

8 


3 

6 . 
9 



We cannot overemphasize the fact that (1) and (2) represent entirely different 
animais. (1) represents a matrix, a rectangular array, an entity unto itself while (2) 
represents a scalar, a number associated with the matrix in (1). There is absolutely 
no similarity between the two other than form! 

We are now ready to calculate déterminants. 


Définition 1 The déterminant of a 1 x 1 matrix [a\ is the scalar a. 

Thus, the déterminant of the matrix [5] is 5 and the déterminant of the matrix 
[-3] is -3. 
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Définition 2 The déterminant of a 2 x 2 matrix 


a b 
c d 


is the scalar ad — bc. 


Example 1 Find det(A) if 



2 

3 


Solution 


det(A) = 


1 

4 


2 

3 


= (1)(3) — (2) (4) = 3 — 



—5. ■ 


Example 2 Find |A| if 




Solution 



(2) (3) — (— 1)(4) = 6 + 4 = 10. ■ 


We now could proceed to give separate rules which would enable one to com- 
pute déterminants of 3 x 3, 4 x 4, and higher order matrices. This is unnecessary. 
In the next section, we will give a method that enables us to reduce ail détermi- 
nants of order n(n >2) (if A has order n x n then det(A) is said to hâve order n) 
to a sum of déterminants of order 2. 


Problems 5.1 

In Problems 1 through 18, find the déterminants of the given matrices. 


"3 

4 

2. 

'3 

-4 

3. 

3 

4" 

5 

6 

5 

6 

-5 

6 

"5 

6" 

5. 

5 

6" 

6. 

"5 

6" 

7 

8 

7 

8 

7 

-8 



5.1 Introduction 


151 




19. Find t so that 


20. Find t so that 


21. Find X so that 


22. Find À so that 


t 

1 




4 — À 

-1 


2 

1 — X 



1 — X 
1 


5 

—1 — X 



23. Find det(A — Àl) if A is the matrix defined in Problem 1. 

24. Find det(A — XI) if A is the matrix defined in Problem 2. 

25. Find det(A — XI) if A is the matrix defined in Problem 4. 

26. Find det(A — XI) if A is the matrix defined in Problem 7. 

27. Find |A|, |B|,and |AB| if 



and B = 




What is the relationship between these three déterminants? 

28. Interchange the rows for each of the matrices given in Problems 1 through 
15, and calculate the new déterminants. How do they compare with the 
déterminants of the original matrices? 
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29. The second elementary row operation is to multiply any row of a matrix 
by a nonzero constant. Apply this operation to the matrices given in Prob- 
lems 1 through 15 for any constants of your choice, and calculate the new 
déterminants. How do they compare with the déterminants of the original 
matrix? 

30. Redo Problem 29 for the third elementary row operation. 

31. What is the déterminant of a 2 x 2 matrix if one row or one column contains 
only zéro entries? 

32. What is the relationship between the déterminant of a 2 x 2 matrix and its 
transpose? 

33. What is the déterminant of a 2 x 2 matrix if one row is a linear combination 
of the other row? 


5.2 Expansion by Cofactors 

Définition 1 Given a matrix A, a minor is the déterminant of any square 
submatrix of A. 


That is, given a square matrix A, a minor is the déterminant of any matrix 
formed from A by the removal of an equal number of rows and columns. As an 
example, if 


then 



1 2 
4 5 
7 8 


3 

6 

9 


1 2 
7 8 


and 


5 6 
8 9 


are both minors because 


1 2 
7 8 


and 


5 6 
8 9 


are both submatrices of A, while 


1 2 
8 9 


and 


1 2 


are not minors because 


1 2 
8 9 


is not a submatrix of A and [1 2], although a submatrix of A, is not square. 
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A more useful concept for our immédiate purposes, since it will enable us to 
calculate déterminants, is that of the cofactor of an element of a matrix. 

Définition 2 Given a matrix A =[a; 7 ], the cofactor of the element aij is a 
scalar obtained by multiplying together the term (— l) 1 + ■> and the minor obtained 
from A by removing the ith row and j th column. 

In other words, to compute the cofactor of the element aij we first form a 
submatrix of A by Crossing out both the row and column in which the element ay 
appears. We then find the déterminant of the submatrix and finally multiply it by 
the number (— 1) I + C 


Example 1 Find the cofactor of the element 4 in the matrix 



1 2 
4 5 
7 8 


3 

6 . 
9 


Solution We first note that 4 appears in the (2, 1) position. The submatrix 
obtained by Crossing out the second row and first column is 



L 

2 

3" 

i 

L- 

s 

_6 


T 

1 

00 L 

KJ 

9_ 


2 3 
8 9 


which has a déterminant equal to (2) (9) — (3) (8) = —6. Since 4 appears in the (2, 1) 
position, i — 2 and j = 1. Thus, (— 1) Z+J = (— 1) 2+1 = (— l) 3 = (—1). The cofactor 
of 4 is (—!)(— 6) = 6. ■ 


Example 2 Using the same A as in Example 1, find the cofactor of the element 9. 


Solution The element 9 appears in the (3, 3) position. Thus, Crossing out the 
third row and third column, we obtain the submatrix 


"1 

2 

/■ 


4 

5 

( 

) 

HL- 

O 


) 

7 

O 


f 


1 2 
4 5 ‘ 


which has a déterminant equal to (1) (5) — (2) (4) = —3. Since, in this case, i — j — 
3, the cofactor of 9 is (— 1) 3 + 3 (— 3) = ( — l) 6 ( — 3) = —3. ■ 


We now hâve enough tools at hand to find the déterminant of any matrix. 

Expansion by Cofactors. To find the déterminant of a matrix A of arbitrary order, 
(a) pick any one row or any one column of the matrix (dealer’s choice), (b) for 
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each element in the row or column chosen, find its cofactor, (c) multiply each 
element in the row or column chosen by its cofactor and sum the results. This sum 
is the déterminant of the matrix. 


Example 3 Find det(A) if 



5 0 
2 1 . 
-6 4 


Solution In this example, we expand by the second column. 


A| = (5) (cofactor of 5) + (2) (cofactor of 2) + (—6) (cofactor of —6) 


= (5)(-l) 1 + 2 


1 1 
3 4 


+ ( 2 )(— 1) 2 + 2 


3 0 
3 4 


+ (— 6)(-l) 3 + 2 


3 Ol 
-1 1 


= 5(— 1)(— 4 - 3) + (2)(1)(12 - 0) + ( — 6)( — 1)(3 - 0) 


= (—5) (—7) + (2) (12) + (6)(3) = 35 + 24 + 18 = 77. 


Example 4 Using the A of Example 3 and expanding by the first row, find 
det(A). 

Solution 


A| = 3 (cofactor of 3) + 5 (cofactor of 5) + 0(cofactor of 0) 


= (3)(— 1) 1 + 1 


2 1 
6 4 


+ 5(— 1) 1+2 


1 1 
3 4 


+ 0 


= (3) (1) (8 + 6) + (5)(— 1)(— 4 — 3) 


= (3) (14) + (—5) (—7) = 42 + 35 = 77. 


The previous examples illustrate two important properties of the method. First, 
the value of the déterminant is the same regardless of which row or column we 
choose to expand by and second, expanding by a row or column that contains 
zéros significantly reduces the number of computations involved. 


Example 5 Find det(A) if 


10 5 2 
-14 10 
3 0 4 1 
-2113 


A = 
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Solution We first check to see which row or column contains the most zéros and 
expand by it. Thus, expanding by the second column gives 


A| = 0(cofactor of 0) + 4(cofactor of 4) + 0(cofactor of 0) + 1 (cofactor of 1) 


0 + 4(— 1) 2+2 

1 5 2 
3 4 1 

+ 0+l(-l) 4+2 

1 5 2 
-1 1 0 


-2 1 3 


3 4 1 



1 

5 

2 


1 

5 

2 

4 

3 

4 

1 

+ 

-1 

1 

0 


-2 

1 

3 


3 

4 

1 


Using expansion by cofactors on each of the déterminants of order 3 yields 


1 5 2 
3 4 1 
-2 1 3 


and 


Hence, 


= 1 (— 1 ) 


1+1 


4 1 
1 3 


+ 5(— 1) 1+2 


3 1 
■2 3 


+ 2 (— 1) 1+3 


3 4 
-2 1 


= —22 (expanding by the first row) 


1 5 2 
1 1 0 
3 4 1 


= 2 (— 1 ) 


1+3 


1 1 

3 4 


+ 0 + 1 (- 1) 3+3 


1 5 
-1 1 


= —8 (expanding by the third column). 


| A| = 4(— 22) - 8 = -88 - 8 = -96. 


For n x n matrices with n > 3, expansion by cofactors is an inefficient pro- 
cedure for calculating déterminants. It simply takes too long. A more élégant 
method, based on elementary row operations, is given in Section 5.4 for matrices 
whose éléments are ail numbers. 


Problems 5.2 

In Problems 1 through 22, use expansion by cofactors to evaluate the déterminants 
of the given matrices. 



"1 

2 

—2~ 


"3 

2 

-2 


"1 

-2 

-2" 

i. 

0 

2 

3 

2. 

1 

0 

4 

3. 

7 

3 

-3 


0 

0 

-3 


2 

0 

-3 


0 

0 

0 
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"2 0 

-1" 




3 

5 

2 


"1 

-3 

-3" 

4. 

1 1 

1 

•> 


5. 

-1 

0 

4 

6. 

2 

8 

3 


_3 2 

— 3_ 




-2 

2 

7 _ 


4 

5 

0 


"2 1 

_( 

T 



~-l 

3 

3" 


"1 

-3 

-3" 

7. 

3 -1 

' 

1 

1 

8. 

1 

1 

4 

9. 

2 

8 

4 


3 -1 

A 

? 
_ / 



-1 

1 

2_ 


3 

5 

1 


"2 1 

3" 




"-1 

3 

3" 


"1 

2 

-3" 

10. 

3 -1 

2 

•> 


11. 

4 

5 

6 

12. 

5 

5 

1 


_2 3 

5_ 




-1 

3 

3_ 


2 

-5 

-1 


"-4 

0 

0 


1 

3 

2" 


3 

-2 

0" 

13. 

2 - 

1 

0 

14. 

-1 

4 

1 

15. 

1 

1 

2 


3 

1 - 

r 

A 

l 


5 

3 

8 


-3 

4 

1 


-12 12 
10 3-1 

2 2-1 1 ’ 

2 0-3 2 


1 

1 

2 

-2 


"-1 

3 

2 

-2" 

1 

5 

2 

-1 

19. 

1 

-5 

-4 

6 

-2 

-2 

1 

3 

3 

-6 

1 

1 

en 

1 

1 

4 

-1 

8 


3 

-4 

3 

-3 


1 2 1-1 

4 0 3 0 

1 10 5 ’ 

2-2 1 1 


20 . 


1 

1 

■2 

■3 


1 

5 


0 

0 


2 0 
4 0 


-2 

-1 

3 

8 


21 . 


16. 


■4 

1 

2 

3 


0 

■5 

1 

1 


0 

0 

■2 

■2 


0 

0 

0 

1 


17. 



11 10 9 0 

2 110 0 

4-1100. 
3 2 2 1 0 

0 0 12 0 


23. Use the results of Problems 1, 13, and 16 to develop a theorem about the 
déterminants of triangular matrices. 

24. Use the results of Problems 3, 20, and 22 to develop a theorem regarding 
déterminants of matrices containing a zéro row or column. 

25. Find det(A — kl) if A is the matrix given in Problem 2. 

26. Find det(A — kl) if A is the matrix given in Problem 3. 

27. Find det(A — kl) if A is the matrix given in Problem 4. 

28. Find det(A — kl) if A is the matrix given in Problem 5. 
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5.3 Properties of Déterminants 

In this section, we list some useful properties of déterminants. For the sake of 
expediency, we only give proofs for déterminants of order three, keeping in mind 
that these proofs may be extended in a straightforward manner to déterminants 
of higher order. 

Property 1 Ifone row ofa matrix consists entirely of zéros, then the déterminant 
is zéro. 

Proof. Expanding by the zéro row, we immediately obtain the desired resuit. 

□ 

Property 2 If two rows of a matrix are interchanged, the déterminant changes 
sign. 

Proof. Consider 



011 

012 

013 

021 

022 

023 

031 

032 

033 


Expanding by the third row, we obtain 

| A| = 031(012 023 — 013 022) — 032(011 023 — 013 «21 ) 
+ 033(011 022 “ 012 021 )• 


Now consider the matrix B obtained from A by interchanging the second and third 
rows: 



011 

012 

013 

031 

032 

033 

021 

022 

023 


Expanding by the second row, we find that 


|B| = —03l(012 023 “ 013 022) + 032(011 023 — 013 02l) 

- 033(011 022 - 012 021 )• 

Thus, |B| = — |A|. Through similar reasoning, one can demonstrate that the resuit 
is valid regardless of which two rows are interchanged. □ 

Property 3 If two rows ofa déterminant are identical, the déterminant is zéro. 
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Proof. If we interchange the two identical rows of the matrix, the matrix remains 
unaltered; hence the déterminant of the matrix remains constant. From Property 
2, however, by interchanging two rows of a matrix, we change the sign of the 
déterminant. Thus, the déterminant must on one hand remain the same while on 
the other hand change the sign. The only way both of these conditions can be met 
simultaneously is for the déterminant to be zéro. □ 

Property 4 If the matrix B is obtained from the matrix A by multiplying every 
element in one row of A by the scalar X, then |B| = X\ A|. 

Proof. 


Xa\\ 

Xa\2 

À<2i3 

<221 

a 22 

<223 

<231 

a 32 

<233 


— À <2 11 


<222 

a 32 


<223 

a 33 


— Xa \2 


a 21 

<231 


a 23 

a 33 


+ ha\3 


<212 

<231 


a 22 

a 32 




<222 

<232 


<223 

<233 


<212 


<221 

<231 


<223 

<233 


+ <213 


<221 

<231 



<211 

<212 

<213 

<221 

<222 

<223 

<231 

<232 

<233 


In essence, Property 4 shows us how to multiply a scalar times a déterminant. 
We know from Chapter 1 that multiplying a scalar times a matrix simply multiplies 
every element of the matrix by that scalar. Property 4, however, implies that mul- 
tiplying a scalar times a déterminant simply multiplies one row of the déterminant 
by the scalar. Thus, while in matrices 


"1 

2" 


" 8 

16" 

_3 

4 


24 

32 


in déterminants we hâve 


1 

2 


1 

2 

3 

4 


24 

32 


or alternatively 


8 


1 2 
3 4 

= 4(2) 

1 2 
3 4 

= 4 

2 4 

3 4 

— 

2 4 

12 16 


□ 


Property 5 For an n x n matrix A and any scalar X, det(ÀA) = X n det(A). 
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Proof. This proof makes continued use of Property 4. 




«Il «12 «13 




Xa\\ ha \2 hai 2 


det(AA) = det 

À 

«21 «22 «23 


= det 


Xa2l À ^22 ^-«23 




_«31 «32 « 33 _ 




_^«31 ^-«32 ^« 33 _ 




A ^12 

À<213 


«11 

«12 

«13 

A .«21 

A «22 

Xa23 

= À 

A. <221 

Xa22 

A.«23 

À«3i 

À«32 

À«33 


À«3i 

À«32 

^«33 



«11 

«12 

«13 


«11 

«12 

«13 

( A ) ( À ) 

«21 

«22 

«23 

= À ( À )( À ) 

«21 

«22 

«23 


À « 3 i 

À «32 

À «33 


«31 

«32 

«33 


= À 3 det(A). 

Note that for a 3 x 3 matrix, n — 3. □ 

Property 6 If a matrix B is obtained from a matrix A by adding to one row of 
A, a scalar times another row of A, then \ A\ — |B|. 

Proof. Let 


and 



«11 

«12 

«13 

«21 

«22 

«23 

«31 

«32 

«33 



a il 
«21 

a 31 + À«n 


a \2 «13 

«22 «23 

«32 + ^«12 ax 3 + ^«13 


where B has been obtained from A by adding À times the first row of A to the 
third row of A. Expanding |B| by its third row, we obtain 


B| = («31 + ’kan) 


«12 

«22 


«13 

«23 


(«32 + A « i 2 ) 


«H 

«21 


«13 

«23 


+ («33 + À « 13 ) 


«11 

«21 


«12 

«22 


= «31 


«12 

«22 


«13 

«23 


- «32 


«11 

«21 


«13 

«23 


+ «33 


«11 

«21 


«12 

«22 


+ À 


«11 


«12 

«22 


«13 

«23 


«12 


«11 

«21 


«13 

«23 


+ «13 


«11 

«21 


«12 

«22 
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The first three terms of this sum are exactly | A| (expand | A| by its third row), while 
the last three terms of the sum are 


a il 

<312 

<313 

<321 

<322 

<323 

an 

<312 

<313 


(expand this déterminant by its third row). Thus, it follows that 


B = A +À 


<311 

<312 

<313 

<321 

<322 

<323 

<311 

<312 

<313 


From Property 3, however, this second déterminant is zéro since its first and third 
rows are identical, hence |B| = |A|. 

The same type of argument will quickly show that this resuit is valid regardless 
of the two rows chosen. □ 


Example 1 Without expanding, show that 


a b 

c 


a — r b — s c — t 

r s 

t 

— 

r + 2x s + 2y t + 2z 

x y 

Z 


x y z 


Solution Using Property 6, we hâve that 


a 

b 

c 


a — r 

b — s 

c — t 

r 

s 

t 

— 

r 

s 

t 

x 

y 

z 


x 

y 

z 


a — r b — s c — t 
r + 2x s + 2y t + 2z 
x y z 


by adding to the first 
row (— 1 ) times the 
second row 

by adding to the 

second row ( 2) times the ■ 

third row 


Property 7 det(A) = det(A T ). 
Proofi If 


<311 

<312 

<313 

, then A t = 

<311 

<321 

<331 

<321 

<322 

<323 

<312 

<322 

a 32 

_<331 

<332 

<333 _ 


_<313 

<323 

<333 _ 


A = 
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Expanding det(A T ) by the first column, it follows that 


a T 


<222 

«32 


<221 

<231 

+ <213 

<221 

<231 

A 

= <211 

<223 

«33 

«12 

«23 

<233 

«22 

<232 


= «Il («22 <233 — <232 < 223 ) — < 212 (< 221<233 — <231 < 223 ) + <213 (<221 <232 — <231 < 222 )* 

This, however, is exactly the expression we would obtain if we expand det(A) by 
the first row. Thus |A t | = |A|. □ 

It follows from Property 7 that any property about déterminants dealing with 
row operations is equally true for column operations (the analogous elementary 
row operation applied to columns), because a row operation on A T is the same 
as a column operation on A. Thus, if one column of a matrix consists entirely of 
zéros, then its déterminant is zéro; if two columns of a matrix are interchanged, the 
déterminant changes the sign; if two columns of a matrix are identical, its détermi- 
nant is zéro; multiplying a déterminant by a scalar is équivalent to multiplying one 
column of the matrix by that scalar and then calculating the new déterminant; and 
the third elementary column operation when applied to a matrix does not change 
its déterminant. 

Property 8 The déterminant of a triangular matrix, either upper or lower, is the 
product ofthe éléments on the main diagonal. 

Proofi See Problem 2. □ 

Property 9 If A and B are ofthe same order, then det(A) det(B) = det(AB). 
Because of its difficulty, the proof of Property 9 is omitted here. 


Example 2 Show that Property 9 is valid for 



3 

4 


and 




Solution |A| = 5, |B| = 31. 


AB = 


33 

34 


10 

15 


thus AB = 155 = A B . ■ 


Problems 5.3 

1. Prove that the déterminant of a diagonal matrix is the product of the éléments 
on the main diagonal. 
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2. Prove that the déterminant of an upper or lower triangular matrix is the 
product of the éléments on the main diagonal. 

3. Without expanding, show that 


a 4- x 

r — 

X 

X 


a 

r 

X 

b 4- y 

s — 

y 

y 

— 

b 

s 

y 

c 4- z 

t - 

z 

z 


c 

t 

z 


4 . Verify Property 5 for À = — 3 and 



0 

3 . 
1 


5 . Verify Property 9 for 



1 

2 


and 




6. Without expanding, show that 


2 a 

3 r 

x 


a 

r 

x 

4 b 

6s 

2 y 

= -12 

b 

s 

y 

-2c 

-3 1 

—z 


c 

t 

z 


7 . Without expanding, show that 


a — 3b r — 3s x — 3 y 


a 

r 

x 

b — 2c s — 2t y — 2z 

= 5 

b 

s 

y 

5c 5 1 5z 


c 

t 

z 


8. Without expanding, show that 


a 

b 

c 


a 

x 

r 

r 

s 

t 

— — 

b 

y 

s 

x 

y 

z 


c 

z 

t 


9 . Without expanding, show that 


abc 
r s t 

x y z 



2 a 4 b 2c 

—r —2s —t 
x 2 y z 


10 . Without expanding, show that 


a — 3x b — 3y 
a 5x b + 5y 
x y 


c — 3z 
c -h 5z 

z 
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11. Without expanding, show that 

2 a 3 a c 
2 r 3 r t = 0. 

2x 3x z 

12 . Prove that if one column of a square matrix is a linear combination of another 
column, then the déterminant of that matrix is zéro. 

13 . Prove that if A is invertible, then det(A _1 ) = 1/ det(A). 

5.4 Pivotai Condensation 

Properties 2, 4, and 6 of the previous section describe the effects on the détermi- 
nant of a matrix of applying elementary row operations to the matrix itself. They 
comprise part of an efficient algorithm for calculating déterminants of matrices 
whose éléments are numbers. The technique is known as pivotai condensation : A 
given matrix is transformed into row-reduced form using elementary row opera- 
tions. A record is kept of the changes to the déterminant as a resuit of Properties 2, 
4, and 6. Once the transformation is complété, the row-reduced matrix is in upper 
triangular form, and its déterminant is found easily by Property 8. In fact, since 
a row-reduced matrix has either unity éléments or zéros on its main diagonal, its 
déterminant will be unity if ail its diagonal éléments are unity, or zéro if any one 
diagonal element is zéro. 

Example 1 Use pivotai condensation to evaluate 

1 2 3 

-2 3 2. 

3 -1 1 

Solution 


1 

2 

3 

1 

2 

3 

Property 6: adding to 

-2 

3 

2 = 

0 

7 

8 

the second row (2) 

3 

-1 

1 

3 

-1 

1 

times the first row 

, 




1 

2 

3 

Property 6: adding to 




0 

7 

8 

the third row (—3) 




0 

-7 

-8 

times the first row 

, 




Property 4: applied 
to the second row 




164 


Chapter 5 Déterminants 


12 3 Property 6: adding 

= 70 1 j to the third row (7) 

0 0 0 times the second row 


= 7(0) = 0. { Property 8 ■ 


Example 2 Use pivotai condensation to evaluate 

0- 1 4 

1- 5 1 . 

-6 2 -3 

Solution 

Property 2: interchanging 
the first and second rows 

Property 6: adding 
to the third row (6) 
times the first row 

, 

Property 4: applied 
to the second row 

Property 6: adding 
to the third row (28) 
times the second row 

Property 4: applied 
to the third row 

= (— 109)(1) = -109. {Property 8 ■ 

Pivotai condensation is easily coded for implémentation on a computer. 
Although shortcuts can be had by créative individuals evaluating déterminants 
by hand, this rarely happens. The orders of most matrices that occur in practice 
are too large and, therefore, too time consuming to consider hand calculations in 
the évaluation of their déterminants. In fact, such déterminants can bring com- 
puter algorithms to their knees. As a resuit, calculating déterminants is avoided 
whenever possible. 

Still, when déterminants are evaluated by hand, appropriate shortcuts are 
taken, as illustrated in the next two examples. The general approach involves 
operating on a matrix so that one row or one column is transformed into a new 


0 

-1 

4 

1 

-5 

1 

1 

-5 

1 =(-l) 

0 

-1 

4 

-6 

2 

-3 

-6 

2 

-3 




1 

-5 

1 



= (-D 

0 

1 

4 




0 

-28 

3 


1 -5 1 

= (— 1)(— 1) 0 1-4 

0 -28 3 

1 -5 1 

= 01 -4 

0 0-109 

1 -5 1 

= (-109) 0 1 -4 

0 0 1 
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row or column containing at most one nonzero element. Expansion by cofactors 
is then applied to that row or column. 


Example 3 Evaluate 


10 -6 -9 
6 -5 -7 . 
-10 9 12 


Solution 


10 

-6 

-9 


10 

-6 

-9 


6 

-5 

-7 

— 

6 

-5 

-7 

* 

10 

9 

12 


0 

3 

3 






10 

-6 

-3 





— 

6 

-5 

-2 

- 





0 

3 

0 



by adding (1) times the 
first row to the third row 
(Property 6) 

by adding (—1) times the 
second column to the 
third column (Property 6) 




jby expansion by cofactors 


= — 3(— 20 + 18) = 6. ■ 


Example 4 Evaluate 


3 

-1 

0 

2 

0 

1 

4 

1 

3 

-2 

3 

5 

9 

7 

0 

2 


Solution Since the third column already contains two zéros, it would seem 
advisable to work on that one. 


3 

-1 

0 

2 


3 

-1 

0 

2 


0 

1 

4 

1 


0 

1 

4 

1 


3 

-2 

3 

5 

— 

3 

il 

4 

0 

17 

4 

* 

9 

7 

0 

2 


9 

7 

0 

2 



= -4 


3 

3 

9 


-1 

il 
' 4 


17 
4 

7 2 


4 Ü 


3-12 
12 -11 17 
9 7 2 


the second row to 
the third row. 


by expansion 
by cofactors 


{by Property 4 
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= (-D 


= (-D 


3-12 
0-7 9 
9 7 2 

3-1 2 

0-7 9 

0 10 -4 


by adding (—4) times 
the first row to the 
second row 

, 

by adding (—3) times 
the first row to the 
third row 


= (- 1 ) 0 ) 


-7 9 

10 -4 


= (-3X28- 90) = 186. 


f by expansion by 
I cofactors 


Problems 5.4 


In Problems 1 through 18, evaluate the déterminants of the given matrices. 


1. 


'1 2 - 2 ' 
1 3 3 


2 . 


'1 

4 


2 

5 


3' 

6 


3. 


3 -4 
1 5 


2 ' 

7 



[2 5 

0 




L 7 

8 9 J 






L 1 

9 

-6 


~-l 3 

3 " 




"1 

-3 - 

/ 

« 

3 " 




"2 

1 

-9" 

4. 

1 1 

4 

9 


5. 

2 

8 

L 

\ 

9 


6. 

3 

-1 

1 


-1 1 

2 




_3 

5 


1 




3 

-1 

2 


"2 1 

3“ 




'-1 

3 : 

3 " 





"1 

2 

-3" 

7. 

3 -1 

2 

9 


8. 

4 

5 ( 

r 

3 

9 



9. 

5 

5 

1 


2 3 

5 




-1 

3 : 

3 





2 

-5 

-1 


"2 0 

-r 




3 

5 2 





"1 

-3 

- 3 " 

10. 

1 1 

i 

9 


11. 

-1 

0 4 

9 



12. 

2 

8 

3 


3 2 

-3 




-2 

2 7 _ 





4 

5 

0 


3 

5 4 6 ' 



"-1 

2 


1 

2 " 






13. 

-2 

1 0 7 


14. 

1 

0 

3 

-1 






-5 

4 7 2 

9 

2 

2 - 

— 

1 

1 

9 






8 - 

■3 : 

1 lj 



2 

0 - 

-3 

2 







1 

1 

2 - 

- 2 “ 


"-1 

3 


2 -2 





15. 

1 

5 

2 - 

-1 

16. 

1 

-5 


-4 ( 

5 

















-2 - 

■2 

1 

3 


3 

-6 



L 

1 

9 





-3 

4 

-1 

8 


3 

-4 


/■ 

3 -3 






17. 


1 

1 

■2 

■3 


1 0 
5 0 
2 0 
4 0 


- 2 ' 

-1 

3 

8 


18. 


2 0 
4 0 


■3 

5 


1 

4 


1 

2 

0 

1 


3' 

-2 

1 

7 
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19. What can you say about the déterminant of an n x n matrix that has rank less 
than ni 

20. What can you say about the déterminant of a singular matrix? 


5.5 Inversion 

As an immédiate conséquence of Theorem 1 of Section 3.2 and the method of 
pivotai condensation, we hâve: 

Theorem 1 A square matrix has an inverse if and only if its déterminant is not 
zéro. 

In this section, we develop a method to calculate inverses of nonsingular matri- 
ces using déterminants. For matrices with order greater than 3x3, this method is 
less efficient than the one described in Section 3.2, and is generally avoided. 

Définition 1 The co factor matrix associated with an n x n matrix A is an n x n 
matrix A c obtained from A by replacing each element of A by its cofactor. 


Example 1 Find A c if 


Solution 


A c = 


(-D 


(-D 


(-D 


1+1 


2+1 


3+1 


A = 


5 4 
3 6 

1 2 
3 6 

1 2 
5 4 


' 312 ' 
-2 5 4 
1 3 6 


(-D 


1+2 


(-D 


2+2 


(-D 


3+2 


2 4 
1 6 

3 2 
1 6 

3 2 
-2 4 


(-D 


1+3 


-2 5 
1 3 


(-D 


2+3 


3 1 
1 3 


(-D 


3+3 


3 1 
-2 5 


A c = 


' 18 16 -11' 
0 16 -8 
-6 -16 17 


If A = \aij\ , we will use the notation A c = [a^] to represent the cofactor matrix. 
Thus a^j represents the cofactor of aq. 

Définition 2 The adjugate of an n x n matrix A is the transpose of the cofactor 
matrix of A. 
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Thus, if we designate the adjugate of A by A a , we hâve that A a = (A c ) 



Example 2 Find A a for the A given in Example 1. 

Solution 



~ 18 

0 

-6 


A° = 

16 

16 

-16 

. ■ 


-11 

-8 

17 



L ™ 





The importance of the adjugate is given in the folio wing theorem, which is 
proved in the Final Comments to this chapter. 

Theorem 2 AA a — A a A = |A|I. 

If | A| 7 ^ 0, we may divide by it in Theorem 2 and obtain 




I. 


Thus, using the définition ofthe inverse, we hâve 

A -1 = -h A» if |A|^0. 

|A| 

That is, if | A| ^ 0, then A -1 may be obtained by dividing the adjugate of A by the 
déterminant of A. 


Example 3 Find A 1 for the A given in Example 1. 


Solution The déterminant of A is found to be 48. Using the solution to 
Example 2, we hâve 





18 

0 

-6" 


3/8 

0 

-1/8 

1/48 

16 

16 

-16 

— 

1/3 

1/3 

— 1/3 


-11 

-8 

17 


-11/48 

-1/6 

17/48 



5 8 
0 2 
4 3 


1 

1 . 
-1 


Example 4 Find A 1 if 
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Solution det(A) = —1^0, therefore A 1 exists. 




-5 4 -8 

11 -9 17 

6 -5 10 


A" = (A f ) T 



-5 

h 

4 

-9 

-8 

17 



Example 5 Find A" 1 if 



2 

4 * 


Solution |A| = —2, therefore A 1 exists. 



Problems 5.5 


In Problems 1 through 15, find the inverses of the given matrices, if they exist. 



4 

4 ’ 


2 . 


1 f 

3 4 




5 


4. 

7. 

10 . 


2 

3 

"1 

1 

0 

"1 

4 
7 

"2 

3 

5 


-1 

4 y 

1 o 
o 1 

î î 

2 3' 
5 6 
8 9 

4 

-4 

0 



5. 

8 

5 

3 

2 

5 


6. 

2 

4 

-1 

-2 

•> 



"0 

0 

r 



"2 

0 

- 1 " 

5 

8. 

1 

0 

0 

5 

9. 

0 

1 

2 



0 

1 

0 



3 

1 

1 



"2 

0 

0" 



"1 

2 

1 

5 

11. 

5 

1 

0 

1 

12. 

3 

-2 

-4 



4 

1 

î 



2 

3 

-1 

3" 


"5 


0 

-r 


"3 

1 

r 

-4 

14. 

2 

— 

î 

2 

, 15. 

1 

3 

-i 

-1 


2 


3 

-1 


2 

3 

-i 


5 


13. 
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16. Find a formula for the inverse of 



b 

d 


if its déterminant is nonzero. 

17. Prove that if A and B are square matrices of the same order, then the product 
AB is nonsingular if and only if both A and B are. 

18. Prove Theorem 1. 

19. What can be said about the rank of a square matrix having a nonzero 
déterminant? 


5.6 Cramer's Rule 

Cramer’s rule is a method, based on déterminants, for solving Systems of simul- 
taneous linear équations. In this section, we first State the rule, then illustrate its 
usage by an example, and finally prove its validity using the properties derived in 
Section 5.3. We also discuss the many limitations of the method. 

Cramer’s rule States that given a System of simultaneous linear équations in 
the matrix form Ax = b (see Section 1.3), the ith component of x (or equivalently 
the ith unknown) is the quotient of two déterminants. The déterminant in the 
numerator is the déterminant of a matrix obtained from A by replacing the zth 
column of A by the vector b, while the déterminant in the denominator is just | A| 
Thus, if we are considering the System 


a \\ x \ + ^12^2 + <213x3 = Z?i, 

<221x1 + <222x2 + <223x3 = b2, 

<231x1 + Æ32X2 + <233x3 = £3, 

where xi, X2, and X3 represent the unknowns, then Cramer’s rule States that 


b\ 

b>2 

b?, 

<212 

<222 

<^32 

<213 

<223 

<233 

_ 

an 

<221 

<231 

b\ 

b>2 

b 3 

<213 

<223 

<233 




|A| 


? X2 — 


|A| 


5 


<211 

<212 

b\ 







<221 

a 22 

b 2 







<231 

<232 

b?, 




<211 

<212 

a 13 


1 A 1 


where | A 

M = 

<221 

<222 

<223 


|/V| 





<231 

<232 

<233 


Two restrictions on the application of Cramer’s rule are immédiate. First, the 
Systems under considération must hâve exactly the same number of équations as 
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unknowns to insure that ail matrices involved are square and hence hâve déter- 
minants. Second, the déterminant of the coefficient matrix must not be zéro since 
it appears in the denominator. If | A| = 0 , then Cramer’s rule cannot be applied. 


Example 1 Solve the System 

x + 2 y — 3 z 4 - w — — 5 , 
y + 3 z + w = 6, 
2 x + 3 y + z + w = 4 , 
x + z 4 - vo — 1 . 

Solution 


"1 

2 

-3 

1“ 


X 


~- 5 " 

0 

1 

3 

1 

, x = 

J 

, b = 

6 

2 

3 

1 

1 


Z 


4 

1 

0 

1 

1 


w 


1 


Since | A| = 20 , Cramer’s rule can be applied, and 


— 

■5 

-2 

-3 

1 





1 

-5 - 

-3 

1 



6 

1 

3 

1 





0 

6 

3 

1 



4 

3 

1 

1 





2 

4 

1 

1 



1 

0 

1 

1 


0 

o, y 


1 

1 

1 

1 

20 



20 




20 “ 



20 



” 20 " 

1 

2 

-5 

1 





1 

2 

-3 

-5 



0 

1 

6 

1 





0 

1 

3 

6 



2 

3 

4 

1 





2 

3 

1 

4 



1 

0 

1 

1 


40 

= 2, 

W = - 

1 

0 

1 

1 


-20 



20 



20 



20 



20 


We now dérivé Cramer’s rule using only those properties of déterminants given 
in Section 5 . 3 . We consider the general System Ax = b where 


an a\2 a\2 • • • a\ n 


XI 


h 

Ü 2 l a 22 <323 • • • <32 n 


x 2 


k >2 

<331 <332 <333 • • • < 33 « 

, X = 

X3 

, and b = 

k>3 

_< 3 h 1 a n 2 a n 2 ' ' * <3 mn_ 


_x n _ 


_pn _ 


A = 
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Th en 



«11*! 

«12 

«13 

• • • «1« 


021*1 

«22 

«23 

• • • «2/7 

*i|A| = 

«31*1 

«32 

«33 

«3 « 


««1*1 

««2 

««3 

««« 


(by Property 4 modified to columns 


«11*1 + «12*2 

«12 

«13 

• • • «1« 

«21*1 + «22*2 

«22 

«23 

• • • «2 « 

«31*1 + «32*2 

«32 

• 

«33 

• • • «3 « 

• 

««1*1 T ««2*2 

• 

««2 

««3 

• 

• • • ««« 


by adding (x 2 ) times 
the second column to 
the first column 


a \\ X \ + <712 X 2 + < 713x3 

«12 

«13 

• • • «1« 

<721 X 1 + «22 X 2 + 023 X 3 

«22 

«23 

• • • «2 « 

«31 X 1 + «32 X 2 + «33 X 3 

• 

«32 

• 

«33 

• 

• • • «3 « 

• 

««1*1 T ««2*2 T ««3*3 

• 

• 

««2 

• 

««3 

• • • ««« 


by adding (* 3 ) 
times the third 
column to the 
first column 


«11*1 4- «12*2 4- «13*3 4- • 

* * + «1«*« 

«12 

«13 

«1« 

«21*1 4 - «22*2 4 - «23*3 + * 

• * + «2 «*« 

«22 

«23 

«2« 

«31*1 + «32*2 + «33*3 + * 

• • + «3h*« 

«32 

«33 

• • • «3« 

««1*1 + ««2*2 + ««3*3 + • 

• • 4" «««*« 

««2 

««3 

««« 


by making continued use of Property 6 in the obvious manner. We now note 
that the first column of the new déterminant is nothing more than Ax, and since, 
Ax = b, the first column reduces to b. 


Thus, 



b \ 

«12 

«13 

«1« 


b 2 

«22 

«23 

• ‘ ‘ «2« 

xi A = 

h 

«32 

«33 

• * ‘ «3 « 


bn 

««2 

««3 

««« 


^1 

«12 

• * ‘ « 1 « 


«22 

• ‘ * « 2 « 

^« 

««2 

««« 


A 


x\ = 
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providing |A| ^ 0. This expression is Cramer ’s rule for obtaining x\. A similar 
argument applied to the j th column, instead of the first column, quickly shows 
that Cramer ’s rule is valid for every xj, j = 1,2, . . . , n. 

Although Cramer’s rule gives a systematic method for the solution of simul- 
taneous linear équations, the number of computations involved can become 
awesome if the order of the déterminant is large. Thus, for large Systems, Cramer’s 
rule is never used. The recommended algorithms include Gaussian élimination 
(Section 2.3) and LU décomposition (Section 3.5). 


Problems 5.6 


Solve the following Systems of équations by Cramer’s rule. 


1. 

x + 2 y = —3, 

2. 

2x 4- y = 3, 


3x + y = 1. 


x — y — 6. 

3. 

4a + 2b = 0, 

4. 

3s — 4 1 — 30, 


5a — 3b — 10. 


—2s + 3 1 — —10. 

5. 

2x — 8 y = 200, 

6. 

x 4- y — 2z — 3, 


—x 4 y = 150. 


2x — y + 3z — 2. 

7. 

x T y = 15, 

8. 

3x 4- y 4- z= 4, 


x -\- z — 15, 


x — y 4- 2z — 15, 


y -\- z — 10. 


2x — 2 y — z — 5. 

9. 

x 4- 2y — 2z — —1, 

10. 

2a 4 3b — c — 4, 


2x 4- y 4- z = 5, 


—a — 2 b 4~ c — —2, 


— x + y — z = —2. 


3a — b —2. 

11. 

2x 4- 3y 4- 2z — 3, 

12. 

5 r 4- 8s 4- t = 2, 


3x 4- y 4- 5z — 2, 


2 s 4 - t — — 1, 


7y — 4z — 5. 


4r 4~ 3s — t — 3. 

13. 

x 4-2 y + z + w = 7, 

3x 4- 4y — 2z — 4 w — 13, 

2x 4- y — z 4- w — —4, 




x — 3y 4- 4z 4- 5w — 0. 



Final Comments on Chapter 5 



We shall now prove Theorem 2 of Section 5.5 dealing with the product of a matrix 
with its adjugate. For this proof we will need the following lemma: 


Lemma 1 If each element ofone row of a matrix is multiplied by the co factor of 
the corresponding element of a different row, the sum is zéro. 
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Proof We prove this lemma only for an arbitrary 3 x 3 matrix A where 



(211 

<312 

<313 

<321 

<322 

<323 

<331 

<332 

<333 


Consider the case in which we multiply every element of the third row by the 
cofactor of the corresponding element in the second row and then sum the results. 
Thus, 


(23i (cofactor of a2i) + a^2 (cofactor of (222) + <233 ( cofactor of (223) 


= <331 (- 

l) 3 

<312 

<313 

+ <332 ( l) 4 

<311 <313 

+ <333 ( l) 5 

<311 <312 




<332 

<333 


<331 <333 


<331 <332 


<311 

<312 

<313 






— 

<331 

<332 

<333 

= 0 {from Property 3 , Section 5.3 



<331 

<332 

<333 







Note that this property is equally valid if we replace the word row by the word 
column. 

Theorem 1 AA a = |A|I. 

Proof. We prove this theorem only for matrices of order 3 x 3 . The proof easily 
may be extended to cover matrices of any arbitrary order. This extension is left as 
an exercise for the student. 



<311 

<312 

<313 


a ll 

a 2l 

a 31 

AA“ = 

<321 

<322 

<323 


a 12 

«22 

a 32 


<331 

<332 

<333 


°13 

«23 

a 33 


If we dénoté this product matrix by \bij\ , then 

b\\ = ai\a c n + (212^12 + «13^13» 
b\2 — Cl\ia C 2i + <312^22 + ^13^23’ 

^23 = ^21^31 + <322^32 + a n a 2V 
/?22 = ^21^21 + a 22 a 22 + a 23 a 23> 

etc. 

We now note that b\\ — \ A\ since it is precisely the term obtained when one com- 
putes det(A) by cofactors, expanding by the first row. Similarly, /?22 = | A| since it 
is precisely the term obtained by computing det(A) by cofactors after expanding 
by the second row. It follows from the above lemma that b\2 — 0 and Z?23 = 0 since 
b\2 is the term obtained by multiplying each element in the first row of A by the 
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cofactor of the corresponding element in the second row and adding, while /?23 
is the term obtained by multiplying each element in the second row of A by the 
cofactor of the corresponding element in the third row and adding. Continuing 
this analysis for each bÿ, we find that 



"A 

0 

0 


"1 

0 

0 " 

AA* = 

0 

|A| 

0 

= A 

0 

1 

0 


0 

0 

|A| 


0 

0 

1 


AA* = |A|I. □ 

Theorem 2 A* A = |A|I. 

Proof. This proof is completely analogous to the previous one and is left as an 
exercise for the student. □ 
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Eigenvalues and Eigenvectors 


6.1 Définitions 

Consider the matrix A and the vectors xi, X2, X3 given by 



1 

4 

-1 


"4" 


"3 


"3“ 

A = 

0 

2 

1 

, xi = 

1 

, X 2 = 

2 

, X 3 = 

0 


0 

0 

3 


0 


2 


0 


Forming the products Axi, Ax2, and AX3, we obtain 


But 


hence, 



"8" 


"9" 


"3" 

Axi = 

2 

, Ax 2 = 

6 

, Ax 3 = 

0 


0 


6 


0 


"8 


"9 



"3" 

2 

0 

= 2xi, 

6 

6 

= 3 x 2 , 

and 

0 

0 



Axi = 2xi, 
Ax 2 = 3x 2 , 
AX3 = 1x3. 


That is, multiplying A by any one of the vectors xi , x 2 , or X3 is équivalent to simply 
multiplying the vector by a suitable scalar. 


Définition 1 A nonzero vector x is an eigenvector (or characteristic vector) of a 
square matrix A if there exists a scalar À such that Ax = Àx. Then À is an eigenvalue 
(or characteristic value) of A. 


Thus, in the above example, xi, x 2 , and X3 are eigenvectors of A and 2, 3, 1 are 
eigenvalues of A. 
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Note that eigenvectors and eigenvalues are only defined for square matrices. 
Furthermore, note that the zéro vector can not be an eigenvector even though 
A* 0 = À* 0 for every scalar À. An eigenvalue, however, can be zéro. 


Example 1 Show that 


x — 



is an eigenvector of 



0 5 7 

0 - 12 . 
0 3 1 


Solution 



"0 

5 

7 


"5 


“0 


"5“ 

Ax = 

0 

-1 

2 


0 

— 

0 

= 0 

0 


0 

3 

1 


0 


0 


0 


Thus, x is an eigenvector of A and À = 0 is an eigenvalue. ■ 


Example 2 Is 


an eigenvector of 




2 

4 


? 


Solution 



"1 

2“ 


T 


"3" 

3 

4 


1 


7 


Thus, if x is to be an eigenvector of A, there must exist a scalar À such that Ax = Àx, 
or such that 


3 

7 


= À 


1 

1 


À 

À 


It is quickly verified that no such À exists, hence x is not an eigenvector of A. ■ 
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Problems 6.1 


1. Détermine which of the following vectors are eigenvectors for 


A = 


1 2 " 
-4 7 • 


(b) 


2 9 


-IJ’ 

-4 
-4 ’ 


2 " 

ij’ 

4" 

-4 ’ 


2. What are the eigenvalues that correspond to the eigenvectors found in 
Problem 1? 

3. Détermine which of the following vectors are eigenvectors for 


B = 


2 -4" 

3 -6 ' 


(b) 


3 ’ 


IJ’ 

-4' 
-6 ’ 


(d) 


(h) 


or 


or 


4 . What are the eigenvalues that correspond to the eigenvectors found in 
Problem 3? 

5 . Détermine which of the following vectors are eigenvectors for 


A = 


2 0 - 1 " 
12 1 . 

■10 2 


(a) 0 , 

0 


(b) 1 , 
0 


(c) -2 , 
1 


-3 
(d) 6 

-3 


(f) 0 , 

1 


(h) 1 . 
1 


6 . What are the eigenvalues that correspond to the eigenvectors found ii 
Problem 5? 

7 . Détermine which of the following vectors are eigenvectors for 


A = 
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(a) 

1 

-1 

0 

(b) 

0" 

0 

1 

> (c) 

r 

0 

0 


0 


-1 


-i 


(d) 


'3' 

1 

0 

0 


—i i 

L_ 

—1 ! 

_ 

— i 

"0" 


~r 


0 


i 

. (e) 

0 

(f) 

0 


O 
i 


— i 
O 
1 


8. What are the eigenvalues that correspond to the eigenvectors found in 
Problem 7? 


6.2 Eigenvalues 

Let x be an eigenvector of the matrix A. Then there must exist an eigenvalue À 
such that 


Ax = kx 

(i) 

or, equivalently, 


o 

x 

*< 

1 

3 


or 


(A — Àl)x = 0. 

(2) 

Caution. We could not hâve written (2) as (A — k)x = 0 since the term A — À 
would require subtracting a scalar from a matrix, an operation which is not defined. 
The quantity A — Àl, however, is defined since we are now subtracting one matrix 
from another. 

Define a new matrix 


B = A — kl. 

( 3 ) 

Then (2) may be rewritten as 


Bx = 0, 

( 4 ) 


is a linear homogeneous System of équations for the unknown x. If B has an 
inverse, then we can solve Eq. (4) for x, obtaining x = B -1 0, or x = 0. This resuit, 
however, is absurd since x is an eigenvector and cannot be zéro. Thus, it follows 
that x will be an eigenvector of A if and only if B does not hâve an inverse. But 
if a square matrix does not hâve an inverse, then its déterminant must be zéro 
(Theorem 1 of Section 5.5). Therefore, x will be an eigenvector of A if and only if 


det (A — kl) = 0. 


( 5 ) 


Equation (5) is called the char acteris tic équation of A. The roots of (5) détermine 
the eigenvalues of A. 
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Example 1 Find the eigenvalues of 


Solution 



2 

3 


A — XI — 


"1 

2“ 

— X 

"1 

0 " 


"1 

2“ 


"à 

0 " 

4 

3 

0 

1 


4 

3 


0 

X 


1 - X 2 
~ 4 3 — À. • 

det (A — AI) = (1 — À) (3 — X) — 8 = X 2 — AX — 5. The characteristic équation 
of A is det (A — XI) — 0, or X 2 — AX — 5 = 0. Solving for À, we hâve that X — —1,5; 
hence the eigenvalues of A are X\ = — 1, X 2 = 5. ■ 


Example 2 Find the eigenvalues of 


Solution 




A — XI — 


"1 

-2“ 

— X 

n 

0" 


1 — X 

1 — 

CN 

1 

1 

1 

0 

1 

— 

1 

1 — X 


det (A - Àl) = (1 - X)(l - X) + 2 = X 2 - 2X + 3. 


The characteristic équation is X 2 — 2X + 3 = 0; hence, solving for À by the quadratic 
formula, we hâve that X\ = 1 + \/2 i, X 2 = 1 — \f2i which are eigenvalues of A. 


Note: Even if the éléments of a matrix are real, the eigenvalues may be complex. 


Example 3 Find the eigenvalues of 


Solution 



2 1 
-t 


A — XI — 


t 2 1 

-X 

n 

0" 


2 1 —t 

0 

1 

1 

— 


t-X 
2 1 


2 1 

-t-X 


det (A — XI) — (t — X)(—t — X) — At 2 — X 2 — 5 1 2 . 
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The characteristic équation is A 2 — 5 1 2 — 0 , hence, the eigenvalues are Ai = V5t, 
À 2 = — \/5 1. 

Note: If the matrix A dépends on a parameter (in this case the parameter is t ), 
then the eigenvalues may also dépend on the parameter. ■ 


Example 4 Find the eigenvalues for 




Solution 



"2 

-1 

1 


1 

0 

1 

0 


2- A 

-1 

1 

A — AI = 

3 

-2 

1 

-A 

0 

1 

0 

— 

3 

-2- A 

1 


0 

0 

1 


1 

0 

0 

1 


0 

0 

1 

r-H 


det (A - XI) = (1 - A)[(2 - A)(— 2 — X) + 3] = (1 — A) (A 2 - 1). 

The characteristic équation is (1 — A) (A 2 — 1) = 0; hence, the eigenvalues are 


X\ — X2 — 1, A3 = —1. H 


Note: The roots of the characteristic équation can be repeated. That is, X1—X2 — 
A3 = • • • = Xk. When this happens, the eigenvalue is said to be of multiplicity k. 
Thus, in Example 4, A = 1 is an eigenvalue of multiplicity 2 while, A = — 1 is an 
eigenvalue of multiplicity 1. 

From the définition of the characteristic Equation (5), it can be shown that 
if A is an n x n matrix then the characteristic équation of A is an nth degree 
polynomial in A. It follows from the Fundamental Theorem of Algebra, that the 
characteristic équation has n roots, counting multiplicity. Hence, A has exactly n 
eigenvalues, counting multiplicity. (See Examples 1 and 4). 

In general, it is very difficult to find the eigenvalues of a matrix. First the 
characteristic équation must be obtained, and for matrices of high order this is a 
lengthy task. Then the characteristic équation must be solved for its roots. If the 
équation is of high order, this can be an impossibility in practice. For example, the 
reader is invited to find the eigenvalues of 


10 7 8 7 

7 5 6 5 

8 6 10 9 

7 5 9 10 


For these reasons, eigenvalues are rarely found by the method just given, and 
numerical techniques are used to obtain approximate values (see Sections 6.6 
and 10.4). 
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In Problems 1 through 35, find the eigenvalues of the given matrices. 


1. 


1 2 ' 
1 4 


4. 


7. 


10 . 


13. 


16. 


'3 6' 
9 6 


'3 5' 

5 “ 3 

1 0' 

0 1 


' 2 2 ' 
-1 -2 

0 t 
2 t —t 


22 . 


i i -r 

00 o 

1 2 3 


25. 


28. 


' 3 
1 

-1 

' 3 
-1 
1 


1 

3 

1 

1 

3 

1 


-r 

-î 

5_ 

r 

-î 

3 


2 . 


'2 r 

2 3 


8 . 


3 5' 

■5 -3 


11. 


14. 


o r 
o o 


'4 10' 

9 -5 


23. 


26. 


29. 


'3 0 0' 

2 6 4 

2 3 5_ 

1 2 3' 

2 4 6 

3 6 9 

1 2 r 

2 4 2 

1 2 1 


3. 


'2 3' 
4 6 


5. 

"2 

-r 

6. 

1 

2" 

, 

1 

4 

, v *. 

4 

-1 


9. 


12 . 


15. 


' 2 5' 

-1 -2 

0 0' 

0 0 

'5 10' 

9 -4 



17. 

" 0 2 1 

, 18. 

' 40 

20' 

5 


1 

K) 

•"t- 

i 


—0 

0_ 


19. 

"1 0 3" 
1 2 1 

, 20. 

0 (N 

(N (N 

1 

-r 

2 

21. 

" 2 0 
2 1 

-r 

2 


3 0 1 


-1 0 

2 


-1 0 

2 


24. 


27. 


30. 


'5 -7 7 
4-3 4 
4 -1 2 

10 2 0 ' 
2 4 6 

0 6 K) 

'4 2 r 
2 7 2 
1 2 4 



" 1 

5 

r 



"0 

1 

0" 




" 0 

1 

0" 

31. 

-1 

-1 

î 

5 

32. 

0 

0 

1 



33. 

0 

0 

1 


0 

0 

3_ 



_0 - 

-1 

0_ 




21 

-27 

9_ 


"1 

-1 

0 

0“ 


" 0 

1 


0 

O - 





34. 

3 

5 

0 

0 

, 35. 

0 

0 


1 

0 





0 

0 

1 

5 

0 

0 


0 

1 

• 





_0 

0 - 

-1 

1_ 


—4 

12 

— 

-13 

6_ 
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36. Consider the matrix 


" 0 

1 

0 ••• 

0 

0 

0 

1 

0 

0 

0 

0 ••• 

1 

_-ao 

—ai 

-Ü2 • • * 

—a n —i 


Use mathematical induction to prove that 

det (C — Àl) = ( — 1) ,7 (À” + a n —\X u -(-••• H- ci2 À^ T ^qÀ + ao). 
Deduce that the characteristic équation for this matrix is 

X n H - a n -lk n T ' * * “1“ ( 22 ^ + ( 2 \ X ( 2 q — 0. 

The matrix C is called the companion matrix for this characteristic équation. 

37. Show that if À is an eigenvalue of A, then kX is an eigenvalue of ÀA, where k 
dénotés an arbitrary scalar. 

38. Show that if X ^ 0 is an eigenvalue of A, then 1/À is an eigenvalue of A -1 , 
providing the inverse exists. 

39. Show that if X is an eigenvalue of A, then it is also an eigenvalue of A T . 


6.3 Eigenvectors 

To each distinct eigenvalue of a matrix A there will correspond at least one 
eigenvector which can be found by solving the appropriate set of homogeneous 
équations. If an eigenvalue À/ is substituted into (2), the corresponding eigenvector 
Xj is the solution of 


(A - kiT)Xi = 0. 



Example 1 Find the eigenvectors of 


Solution The eigenvalues of A hâve already been found to be X\ — — 1, X 2 = 5 
(see Example 1 of Section 6.2). We first calculate the eigenvectors corresponding 
to Ài. From (6), 


(A-(— l)I)xi=0. 


( 7 ) 
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If we designate the unknown vector xi by 


x\ 

yij 


Eq. (7) becomes 


or 


or, equivalently, 


1 

2 


1 

0 



x\ 

4 

3 

+ 

0 

1 



yi 


0 

0 


"2 

2 " 


XI 


" 0 " 

4 

4 


_yi_ 


0 


2xi + 2yi = 0, 
4xi + 4yi = 0. 


A nontrivial solution to this set of équations is xi = — yi, y\ arbitrary; hence, the 
eigenvector is 


xi 


xi 

yi 




yi arbitrary. 


By choosing different values of yi, different eigenvectors for X\ = — 1 can be 
obtained. Note, however, that any two such eigenvectors would be scalar mul- 
tiples of each other, hence linearly dépendent. Thus, there is only one linearly 
independent eigenvector corresponding to X\ = —1. For convenience we choose 
yi = 1, which gives us the eigenvector 


xi 



Many times, however, the scalar yi is chosen in such a manner that the resulting 
eigenvector becomes a unit vector. If we wished to achieve this resuit for the above 
vector, we would hâve to choose yi = 1/V2. 

Having found an eigenvector corresponding to X\ = — 1, we proceed to find an 
eigenvector X 2 corresponding to X 2 = 5. Designating the unknown vector X 2 by 


X 2 

J2 


and substituting it with X 2 into (6), we obtain 


"1 2 " 

c 

0 " 

t-H 

1 

4 3 

— J 

0 lj 


X2 

yi 



1 


"-4 

2“ 


x 2 


" 0 " 

4 

-2 


J2_ 


0 


or 
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or, equivalently, 


-4x 2 + 2y 2 = 0, 
4 x 2 - 2y 2 = 0. 


A nontrivial solution to this set of équations is x 2 = \y2, where y 2 is arbitrary; 
hence 



^ 2/2 

yi 




For convenience, we choose y 2 = 2, thus 


x 2 



In order to check whether or not x 2 is an eigenvector corresponding to À 2 = 5, 
we need only check if Ax 2 = À 2 X 2 : 



2“ 


ru 

3 


bj 

i 



= ^2 X 2- 


Again note that x 2 is not unique! Any scalar multiple of x 2 is also an eigenvector 
corresponding to However, in this case, there is just one linearly independent 
eigenvector corresponding to ■ 


Example 2 Find the eigenvectors of 


A = 


'2 

0 

0 


0 

2 

-1 


0 ' 

5 

-2 


Solution By using the method of the previous section, we find the eigenvalues 
to be = 2, À 2 = i, À 3 = —i. We first calculate the eigenvectors corresponding to 
k\ = 2. Designate xi by 


Then ( 6 ) becomes 


xi 

yi 

Zi 



"2 

0 

0" 


"1 

0 

0" 



xi 


"0" 


0 

2 

5 

-2 

0 

1 

0 


• 

yi 

— 

0 

, 

0 

-1 

-2 


0 

0 

1 



J i_ 


0 
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or, equivalently, 


0 0 0 x\ 

0 0 5 yi 

0 -1 -4 zi 


0" 

0 , 
0 


0 = 0 , 

5 zi = 0, 

—yi — 4zi = 0. 

A nontrivial solution to this set of équations is yi = z\ = 0, jq arbitrary; hence 


xi = \yi = 


x\ 1 

0 = x\ 0 

0 0 


We now find the eigenvectors corresponding to X 2 = i. If we designate X 2 by 


Eq. (6) becomes 


J2 b 


2 — i 0 
0 2 — i 

0 -1 


0 

5 

-2 -i 


(2 - i)x 2 = 0, 

(2 - ï)y 2 + 5z 2 = 0, 

-y2 + (-2 - i)z 2 = 0. 

A nontrivial solution to this set of équations is x 2 — 0, y2 = (—2 — ï)z 2 , z 2 arbitrary; 
hence, 

x 2 0 0 

x 2 — yi — (-2 - i)z 2 = Z 2 -2 - i . 

_Z2J [_ Z 2 J |_ 1 

The eigenvectors corresponding to = —i are found in a similar manner to be 


x 3 — Z 3 -2- i , Z3 arbitrary. 
1 
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It should be noted that even if a mistake is made in finding the eigenvalues of 
a matrix, the error will become apparent when the eigenvectors corresponding to 
the incorrect eigenvalue are found. For instance, suppose that in Example 2 was 
calculated erroneously to be 3. If we now try to find xi we obtain the équations. 

-x\ = 0, 

-yi + 5zi = 0, 

-yi - 5 zi = 0 . 


The only solution to this set of équations is x\ = y\ = zi = 0, hence 


xi 


0 

0 . 
0 


However, by définition, an eigenvector cannot be the zéro vector. Since every 
eigenvalue must hâve a corresponding eigenvector, there is a mistake. A quick 
check shows that ail the calculations above are valid, hence the error must lie in 
the value of the eigenvalue. 


Problems 6.3 


In Problems 1 through 23, find an eigenvector corresponding to each eigenvalue 
of the given matrix. 


1. 


1 2 
1 4 


2 . 

"2 r 

3 . 

"2 3" 

5 

2 3 




4 . 


3 6 
9 6 


5 . 


1 2 ' 
4 -1 


6 . 


'3 5' 

5 -3 


7 . 

3 


5 

8 . 

2 

5 



9 . 

2 

2 



5 

— ^ 

3 

-1 

-2 

5 


-1 

-2 

5 


10 . 

"4 

10 " 


11 . 

"0 

t 




12 . 

"4(9 

20 



_9 

— 5 _ 

i 

2 1 

-t 

•> 



-0 

0 _ 

5 



"1 

0 

3 " 


2 

0 


— 



3 

0 

- 1 " 


13 . 

1 

2 

1 

14 . 

2 

2 


2 

, 15 . 

2 

3 

2 

•> 


3 

0 

1 


-1 

0 


2 


-1 

0 

3 



"3 

0 

0 " 


"5 

-7 


7" 



3 

1 

— 

r 

16 . 

2 

6 

4 

17 . 

4 

-3 


4 

•> 

18 . 

1 

3 

— _ 

i 


2 

3 

5 


4 

-1 


2 



-1 

-1 

i 

5 
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1 

5 r 



1 

O 

1 0“ 



"3 

2 

r 

19. 

-1 

-i î 

•> 

20. 

0 

0 1 


21. 

0 

4 

0 


0 

0 3 



0 

-1 0 



0 

1 

5 


"1 

1 

t— 1 

O 

O 


"2 

4 2 - 

- 2 " 





22. 

3 

5 0 0 

23. 

0 

1 0 

0 












0 

0 1 4 

^ Mm ^7 • 

0 

3 3 - 

-1 

• 





0 

0 1 1 


0 

2 0 

4 






24. Find unit eigenvectors (i.e., eigenvectors whose magnitudes equal unity) for 
the matrix in Problem 1. 

25. Find unit eigenvectors for the matrix in Problem 2. 

26. Find unit eigenvectors for the matrix in Problem 3. 

27. Find unit eigenvectors for the matrix in Problem 13. 

28. Find unit eigenvectors for the matrix in Problem 14. 

29. Find unit eigenvectors for the matrix in Problem 16. 

30. A nonzero vector x is a left eigenvector for a matrix A if there exists a scalar X 
such that xA = Xx. Find a set of left eigenvectors for the matrix in Problem 1. 

31. Find a set of left eigenvectors for the matrix in Problem 2. 

32. Find a set of left eigenvectors for the matrix in Problem 3. 

33. Find a set of left eigenvectors for the matrix in Problem 4. 

34. Find a set of left eigenvectors for the matrix in Problem 13. 

35. Find a set of left eigenvectors for the matrix in Problem 14. 

36. Find a set of left eigenvectors for the matrix in Problem 16. 

37. Find a set of left eigenvectors for the matrix in Problem 18. 

38. Prove that if x is a right eigenvector of a symmetric matrix A, then x T is a left 
eigenvector of A. 

39. A left eigenvector for a given matrix is known to be [1 1]. Find another left 

eigenvector for the same matrix satisfying the property that the sum of the 
vector components must equal unity. 

40. A left eigenvector for a given matrix is known to be [2 3]. Find another left 
eigenvector for the same matrix satisfying the property that the sum of the 
vector components must equal unity. 

41. A left eigenvector for a given matrix is known to be [1 2 5]. Find another 
left eigenvector for the same matrix satisfying the property that the sum of 
the vector components must equal unity. 

42. A Markov chain (see Problem 16 of Section 1.1 and Problem 16 of Section 1.6) 
is regular if some power of the transition matrix contains only positive élé- 
ments. If the matrix itself contains only positive éléments then the power 
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is one, and the matrix is automatically regular. Transition matrices that are 
regular always hâve an eigenvalue of unity. They also hâve limiting distribu- 
tion vectors denoted by x (oo \ where the ith component of x (oo) represents the 
probability of an object being in State i after a large number of time periods 
hâve elapsed. The limiting distribution x (00) is a left eigenvector of the tran- 
sition matrix corresponding to the eigenvalue of unity, and having the sum of 
its components equal to one. 

(a) Find the limiting distribution vector for the Markov chain described in 
Problem 16 of Section 1.1. 

(b) Ultimately, what is the probability that a family will résidé in the city? 

43. Find the limiting distribution vector for the Markov chain described in Prob- 
lem 17 of Section 1.1. What is the probability of having a Republican mayor 
over the long run? 

44. Find the limiting distribution vector for the Markov chain described in Prob- 
lem 18 of Section 1.1. What is the probability of having a good harvest over 
the long run? 

45. Find the limiting distribution vector for the Markov chain described in Prob- 
lem 19 of Section 1.1. Ultimately, what is the probability that a person will use 
Brand Y? 


6.4 Properties of Eigenvalues and Eigenvectors 

Définition 1 The trace of a matrix A, designated by tr(A), is the sum of the 
éléments on the main diagonal. 


Example 1 Find the tr(A) if 




Solution tr(A) = 3 + 4 + (—5) —2. ■ 


Property 1 The sum of the eigenvalues of a matrix e quais the trace of the matrix. 


Proof. See Problem 20. □ 

Property 1 provides us with a quick and useful procedure for checking 
eigenvalues. 

Example 2 Verify Property 1 for 
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Solution The eigenvalues of A are X\ = 10, X 2 = —4. 

tr(A) = 11 + (—5) = 6 = X\ + À 2 . ■ 


Property 2 A matrix is singular if and only ifit has a zéro eigenvalue. 

Proof. A matrix A has a zéro eigenvalue if and only if det(A — 01) = 0, or (since 
01 = 0) if and only if det(A) = 0. But det(A) = 0 if and only if A is singular, thus, 
the resuit is immédiate. □ 

Property 3 The eigenvalues of an upper (or lower) triangular matrix are the 
éléments on the main diagonal. 

Proof. See Problem 15. □ 


Example 3 Find the eigenvalues of 



1 0 
2 1 
3 4 


0 

0 . 
-1 


Solution Since A is lower triangular, the eigenvalues must be X\ = X^ = 1, 
*3 = -l- ■ 


Property 4 If X is an eigenvalue of A and if A is invertible, then 1/X is an 
eigenvalue of A -1 . 

Proof. Since A is invertible, Property 2 implies that À ^ 0; hence 1/X exists. 
Since X is an eigenvalue of A there must exist an eigenvector x such that Ax = Xx. 
Premultiplying both sides of this équation by A -1 , we obtain 

x = AA -1 x 

or, equivalently, A -1 x = (1/À)x. Thus, 1/X is an eigenvalue of A -1 . □ 

Observation 1 If x is an eigenvector of A corresponding to the eigenvalue X 
and if A is invertible, then x is an eigenvector of A -1 corresponding to the 
eigenvalue 1/X. 

Property 5 IfX is an eigenvalue of A, then aX is an eigenvalue of a A where a is 
any arbitrary scalar. 

Proof. If À is an eigenvalue of A, then there must exist an eigenvector x such that 
Ax = Xx. Multiplying both sides of this équation by a , we obtain (û'A)x = (qA)x 
which implies Property 5. □ 
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Observation 2 If x is an eigenvector of A corresponding to the eigenvalue À, 
then x is an eigenvector of a A corresponding to eigenvalue aX. 

Property 6 If X is an eigenvalue of A, then X k is an eigenvalue of A k , for any 
positive integer k. 

Proof. We prove the resuit for the spécial cases k — 2 and k equals 3. Other cases 
are handled by mathematical induction. (See Problem 16.) If À is an eigenvalue 
of A, there must exist an eigenvector x such that Ax = Xx. Then, 

A 2 x = A(Ax) = A(Àx) = À(Ax) = X(Xx) = X 2 x , 

which implies that X 2 is an eigenvalue of A 2 . As a resuit, we also hâve that 

A 3 x = A(A 2 x) = A(À 2 x) = à 2 (Ax) = X 2 (Xx) = X 3 x, 

which implies that À 3 is an eigenvalue of A 3 . □ 

Observation 3 If x is an eigenvector of A corresponding to the eigenvalue À, 
then x is an eigenvector A k corresponding to the eigenvalue X k , for any positive 
integer k. 

Property 7 IfX is an eigenvalue of A, then for any scalar c,X — c is an eigenvalue 
of A — cl. 


Proof. If X is an eigenvalue of A, then there exists an eigenvector x such that 
Ax = Xx. Consequently, 

Ax — ex = Àx — ex, 


or 

(A — cl)x — (X — c)x. 
Thus, X — c is an eigenvalue of A — cl. 


□ 


Observation 4 If x is an eigenvector of A corresponding to the eigenvalue À, 
then x is an eigenvector A — cl corresponding to the eigenvalue X — c. 


Property 8 IfX is an eigenvalue of A, then X is an eigenvalue of A T . 
Proof. Since X is an eigenvalue of A, det(A — AI) = 0. Hence 


0 = 


A - Al| = |(A t ) T -àI t 

(a t - ai) t | 

A t - Ail 


{Property 1 (Section 1.4) 

{Property 3 (Section 1.4) 
{Property 7 (Section 5.3) 


Thus, det (A T — AI) = 0, which implies that X is an eigenvalue of A T . 


□ 
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Property 9 The product of the eigenvalues (counting multiplicity ) of a matrix 
equals the déterminant of the matrix. 

Proof. See Problem 21. □ 

Example 4 Verify Property 9 for the matrix A given in Example 2: 

Solution For thisA, = 10, À 2 = — 4,det(A) = —55 + 15 = —40 = A 4 A. 2 . ■ 


Problems 6.4 


1. One eigenvalue of the matrix 



2 

3 


is known to be 2. Détermine the second eigenvalue by inspection. 

2. One eigenvalue of the matrix 



3 

2 


is known to be 0.7574, rounded to four décimal places. Détermine the second 
eigenvalue by inspection. 

3. Two eigenvalues of a 3 x 3 matrix are known to be 5 and 8. What can be said 
about the remaining eigenvalue if the trace of the matrix is —4? 

4. Redo Problem 3 if the déterminant of the matrix is —4 instead of its trace. 

5. The détermination of a 4 x 4 matrix A is 144 and two of its eigenvalues are 
known to be —3 and 2. What can be said about the remaining eigenvalues? 

6. A 2 x 2 matrix A is known to hâve the eigenvalues —3 and 4. What are the eigen- 
values of (a) 2A, (b) 5A, (c) A — 31, and (d) A + 41? 

7. A3 x 3 matrix A is known to hâve the eigenvalues —2, 2, and 4. What are the 
eigenvalues of (a) A 2 , (b) A 3 , (c) — 3A, and (d) A + 31? 

8. A 2 x 2 matrix A is known to hâve the eigenvalues —1 and 1. Find a matrix in 
terms of A that has for its eigenvalues: 

(a) —2 and 2, (b) —5 and 5, 

(c) 1 and 1, (d) 2 and 4. 

9. A 3x3 matrix A is known to hâve the eigenvalues 2, 3, and 4. Find a matrix 
in terms of A that has for its eigenvalues: 

(a) 4, 6, and 8, (b) 4, 9, and 16, 

(c) 8, 27, and 64, (d) 0, 1, and 2. 
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21 . 

22 . 



Verify Property 1 for 


Verify Property 2 for 



16 

-7 • 



1 3 
-1 2 

2 1 



Show that if À is an eigenvalue of A, then it is also an eigenvalue for S 1 AS 
for any nonsingular matrix S. 


Show by example that, in general, an eigenvalue of A + B is not the sum of 
an eigenvalue of A with an eigenvalue of B. 


Show by example that, in general, an eigenvalue of AB is not the product of 
an eigenvalue of A with an eigenvalue of B. 


Prove Property 3. 

Use mathematical induction to complété the proof of Property 6. 


The déterminant of A — Àl is known as the characteristic polynomial of A. 
For an n x n matrix A, it has the form 


det (A — Àl) — (— l) n (X n + a n —\X n ^ + a n — 2 À ,? ^ + • • • + C12X ^ + a\X + üq), 


where a n - 1 , a n ~ 2 , . . . , <22, a\,ao are constants that dépend on the éléments of 
A. Show that (— l) n ao = det (A). 

(Problem 17 continued) Convince yourself by considering arbitrary 3x3 and 
4x4 matrices that a n -\ = tr(A). 

Assume that A is an n x n matrix with eigenvalues X\, X2 , . . . , X n , where 
some or ail of the eigenvalues may be equal. Since each eigenvalue 
À/(/ = 1,2,..., n) is a root of the characteristic polynomial, (À — À/) must 
be a factor of that polynomial. Deduce that 


det (A - Àl) = (-1) /7 (À - Ài)(À - À 2 ) • • • (À - X n ). 


Use the results of Problems 18 and 19 to prove Property 1. 

Use the results of Problems 17 and 19 to prove Property 9. 

Show, by example, that an eigenvector of A need not be an eigenvector of A T . 
Prove that an eigenvector of A is a left eigenvector of A T . 


6.5 Liriearly Independent Eigenvectors 

Since every eigenvalue has an infinité number of eigenvectors associated with 
it (recall that if x is an eigenvector, then any scalar multiple of x is also an 
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eigenvector), it becomes academie to ask how many different eigenvectors can 
a matrix hâve? The answer is clearly an infinité number. A more revealing ques- 
tion is how many linearly independent eigenvectors can a matrix hâve? Theorem 
4 of Section 2.6 provides us with a partial answer. 

Theorem 1 In an n-dimensional vector space, every set of n + 1 vectors is 
linearly dépendent. 

Therefore, since ail of the eigenvectors of an n x n matrix must be n- 
dimensional (why?), it follows from Theorem 1 that an n x n matrix can hâve 
at most n linearly independent eigenvectors. The following three examples shed 
more light on the subject. 


Example 1 Find the eigenvectors of 


"2 1 0 
A = 0 2 1 
0 0 2 


Solution The eigenvalues of A are X\ = X 2 = X 3 = 2, therefore X — 2 is an 
eigenvalue of multiplicity 3. If we designate the unknown eigenvector x by 


y > 


then Eq. (6) gives rise to the three équations 

y = o, 

£ — 0, 

0 = 0. 

Thus, y = z — 0 and x is arbitrary; hence 


x = m = 


"1 

= v 0 
0 


Setting x 
vector, 


= 1, we see that X — 2 generates only one linearly independent eigen- 


x = 0 . 

0 
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Example 2 Find the eigenvectors of 



2 1 
0 2 
0 0 



Solution Again, the eigenvalues are X\ — X 2 = À 3 = 2, therefore X = 2 is an 
eigenvalue of multiplicity 3. Designate the unknown eigenvector x by 



Equation (6) now gives rise to the équations 

y = o, 

0 = 0, 

0 = 0. 


Thus, y = 0 and both x and z are arbitrary; hence 



Since x and z can be chosen arbitrarily, we can first choose x = 1 and z = 0 to 
obtain 



and then choose x = 0 and z = 1 to obtain 



xi and X2 can easily be shown to be linearly independent vectors, hence we see 
that X = 2 generates the two linearly independent eigenvectors 
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Example 3 Find the eigenvectors of 



0 0 

2 0 . 

0 2 


Solution Again the eigenvalues are = X2 = À 3 = 2 so again X = 2 is an 
eigenvalue of multiplicity three. Designate the unknown eigenvector x by 


x 

J • 

_z_ 

Equation (6) gives rise to the équations 

0 = 0 , 
0 = 0 , 
0 = 0 , 

Thus, x, y, and z are ail arbitrary; hence 



X 


X 


"0 


"0 


"1 


"0 


" 0 " 

X = 

J 

— 

0 

+ 

J 

+ 

0 

= x 

0 


1 

+ Z 

0 


Z 


0 


0 


z_ 


0 


0 


1 


Since x, y, and z can be chosen arbitrarily, we can first choose x = 1, y = z = 0, 
then choose x = z = 0, y = 1 and finally choose y = x = 0, z = 1 to generate the 
three linearly independent eigenvectors 


1 


1 

0 

1 


0" 
1 

0 

1 

1 

1 

0 

1 — 
0 
1 


1 

0 
1 


1 


In this case we see that three linearly independent eigenvectors are generated 
by À = 2. (Note that, from Theorem 1, this is the maximal number that could be 
generated.) ■ 

The preceding examples are illustrations of 

Theorem 2 If X is an eigenvalue of multiplicity k of an n x n matrix A, then 
the number of linearly independent eigenvectors of A associated with X is given by 
p — n — r( A — Àl). Furthermore, l < p < k. 
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Proof Let x be an ft-dimensional vector. If x is an eigenvector, then it must 
satisfy the vector équation Ax = Xx or, equivalently, (A — Àl)x = 0. This System is 
homogeneous, hence consistent, so by Theorem 2 of Section 2.7, we hâve that the 
solution vector x will be in terms of n — r( A — XI) arbitrary unknowns. Since these 
unknowns can be picked independently of each other, it follows that the number of 
linearly independent eigenvectors of A associated with À is also p — n — r( A — XI). 
We defer a proof that 1 < p < k until Chapter 9. □ 

In Example 1, A is 3 x 3 ; hence n — 3, and r(A — 21) = 2. Thus, there should be 
3 — 2 = 1 linearly independent eigenvector associated with X — 2 which is indeed 
the case. In Example 2, once again n — 3 but r(A — 21) = 1. Thus, there should 
be 3 — 1 = 2 linearly independent eigenvectors associated with X — 2 which also 
is the case. 

The next theorem gives the relationship between eigenvectors that correspond 
to different eigenvalues. 

Theorem Eigenvectors corresponding to distinct (that is, different) eigenvalues 
are linearly independent. 

Proof. For the sake of clarity, we consider the case of three distinct eigenvectors 
and leave the more general proof as an exercise (see Problem 17). Therefore, 
let X\, X 2 , A. 3 , be distinct eigenvalues of the matrix A and let xi, X 2 , X 3 be the 
associated eigenvectors. That is 

Ax x = Mxi, 

Ax 2 = À 2 X 2 , ( 8 ) 

Ax 3 = À 3 X 3 , 

and X\ 7 ^ X 2 7 ^ X 3 7 ^ X\. 

Since we want to show that xi, x 2 , X 3 are linearly independent, we must show 
that the only solution to 

C 1 X 1 + C 2 x 2 + C 3 X 3 = 0 (9) 

is c\ — c 2 — c 2 — 0. By premultiplying (9) by A, we obtain 

ciAxi + c 2 Ax 2 + C 3 AX 3 = A *0 = 0. 

It follows from ( 8 ), therefore, that 

c\X\xi + c 2 X 2 x 2 + C 3 À 3 X 3 = 0. (10) 

By premultiplying (10) by A and again using ( 8 ), we obtain 

ci *1X1 + C2À2X2 + <73^3X3 = 0. 

Equations (9)— (11) can be written in the matrix form 


"1 1 1 


cixi 


" 0 " 

X\ X 2 À3 


C 2 X 2 

— 

0 

x\ x\ x\ 


_C 3 X 3 _ 


0 


( 11 ) 
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Define 




À2 A3 # 



It can be shown that det(B) = (A 2 — Ai) (A 3 — A 2 XA 3 — Ai). Thus, since ail the 
eigenvalues are distinct, det (B) 7 ^ 0 and B is invertible. Therefore, 


C2*2 

= B -1 

"0 

0 



" 0 " 

0 

_C 3 X 3 _ 


0 


0 


or 


cixi = 0 
C 2 X2 = 0 
C 3 X 3 = 0 



But since xi, X 2 , X 3 are eigenvectors, they are nonzero, therefore, it folio ws from 
(12) that c\ — C 2 — c 2 — 0. This resuit together with (9) implies Theorem 3. 

Theorems 2 and 3 together completely détermine the number of linearly 
independent eigenvectors of a matrix. □ 


Example 4 Find a set of linearly independent eigenvectors for 



0 0 
3 2 . 
2 3 


Solution The eigenvalues of A are Ai = A2 = 1, and A3 = 5. For this matrix, 
n — 3andr(A — II) = l,hence« — r(A — II) = 2. Thus, from Theorem 2, weknow 
that A has two linearly independent eigenvectors corresponding to A = 1 and one 
linearly independent eigenvector corresponding to A = 5 (why?). Furthermore, 
Theorem 3 guarantees that the two eigenvectors corresponding to A = 1 will be 
linearly independent of the eigenvector corresponding to A = 5 and vice versa. It 
only remains to produce these vectors. 


For A = 1, the unknown vector 


xi 


xi 


yi 
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must satisfy the vector équation (A — ll)xi = 0, or equivalently, the set of 
équations 

0 = 0 , 

4xi + 2yi + 2zi = 0, 

4xi + 2yi + 2zi = 0. 


A solution to this équation is z î = — 2xi — yi, xi, and y\ arbitrary. Thus, 



XI 


XI 


r 


0" 

XI = 

yi 

— 

yi 

= X\ 

0 

+ J 1 

1 


_Z1_ 


2xi - yi_ 


-2 


-1 


By first choosing xi = 1, y\ — 0 and then xi = 0, y\ — 1, we see that À = 1 
generates the two linearly independent eigenvectors 


1 


o~ 
1 

0 

5 

1 

1 

(N 

1 

1 


-1 


An eigenvector corresponding to À 3 = 5 is found to be 

" 0 " 

1 . 

1 

Therefore, A possesses the three linearly independent eigenvectors, 


— 1 
1 


1 

0 

1 


"0" 

0 

•> 

1 

•> 

1 

i 

1 

N) 

1 


-1 


1 



Problems 6.5 


In Problems 1-16 find a set of linearly independent eigenvectors for the given 
matrices. 



2 . 


'3 1' 
0 3 



3 

0 
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13 . 


15 . 


1 

1 

1 

1 


0 1 
0 0 
0 0 
1 4 

0 
2 
1 
1 


0 

1 

2 

1 


0 

1 

0 

-6 

0" 

1 

1 

2 


0' 

0 

1 

4 


14 . 


16 . 


'1 

0 

0 

0 

'3 

0 

0 

0 


0 

0 

0 

1 

1 

3 

0 

0 


0 
1 
0 
-3 

1 2 ' 
1 1 
2 0 
0 2 


0' 

0 

1 

3 



"2 

1 

r 




"2 

i r 



2 


0 - 

-1" 

4 . 

0 

1 

0 

5 


5 . 

0 

1 0 


6 . 

2 


1 - 

-2 


1 

1 

2 




1 

2 2 



-1 


0 

2 


"1 

1 

-1 



"1 

2 3" 



3 


-1 

1 

7 . 

0 

0 

0 

1 

8 . 

2 

4 6 

•> 

9 . 

-1 


3 

-1 


1 

2 

/ 

• 




3 

6 9 



1 


-1 

3 


" 0 


1 


0" 


"0 

1 01 


"4 

2 

r 


10 . 

0 


0 


1 

11 . 

0 

o : 

i 

12 . 

2 

7 

2 

5 


27 


-27 


9 


1 

-3 : 

3 


1 

2 

4 



17 . The Vandermonde déterminant 


1 


x-. 


1 


n — 1 n — 1 


X^ 


1 


XI 

X2 

• • • 

X n 


7 


J2 

X\ 

x 2 

• • • 

x n 




n — 1 


n 


is known to equal the product 


(X2 ~ Xi)(X3 - X2)(X3 - X\)(X4 - X3)(X4 - xi) * * * (x n ~ X\). 


Using this resuit, prove Theorem 3 for n distinct eigenvalues. 


6.6 Power Methods 

The analytic methods described in Sections 6.2 and 6.3 are impractical for calculat- 
ing the eigenvalues and eigenvectors of matrices of large order. Determining the 
characteristic équations for such matrices involves enormous effort, while finding 
its roots algebraically is usually impossible. Instead, itérative methods which lend 



202 


Chapter 6 Eigenvalues and Eigenvectors 


themselves to computer implémentation are used. Ideally, each itération yields 
a new approximation, which converges to an eigenvalue and the corresponding 
eigenvetor. 

The dominant eigenvalue of a matrix is the one having largest absolute val- 
ues. Thus, if the eigenvalues of a matrix are 2, 5, and —13, then —13 is the 
dominant eigenvalue because it is the largest in absolute value. The power 
method is an algorithm for locating the dominant eigenvalue and a correspond- 
ing eigenvector for a matrix of real numbers when the following two conditions 
exist: 

Condition 1. The dominant eigenvalue of a matrix is real (not complex) and is 
strictly greater in absolute values than ail other eigenvalues. 

Condition 2. If the matrix has order n x n, then it possesses n linearly 
independent eigenvectors. 

Dénoté the eigenvalues of a given square matrix A satisfying Conditions 1 
and 2 by Ài, À 2 , . . . , k n , and a set of corresponding eigenvectors by vi, V 2 , . . . , \ n , 
respectively. Assume the indexing is such that 


l^ll > I A2I > | A3 1 >•••>! A« 


Any vector xo can be expressed as a linear combination of the eigenvectors of A, 
so we may write 


XO = Cl VI + C 2 V 2 H h c n \ n . 

Multiplying this équation by A*, for some large, positive integer k, we get 


A a xo = A A '(civi + C2V2 H \~ C n \ n ) 

= ciA a vi + c 2 A a v 2 H h c n A k \ n . 


It follows from Property 6 and Observation 3 of Section 6.4 that 


A a x 0 = c 1 X k \ 1 + c 2 X k \ 2 H + c n X k \ n 


4 


C 1 V 1 + C2 | — V 2 H \-C n ( 7— i Vn 


Ài 


Ai 


O-' 


À A civi for large/:. 
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This last pseudo-equality follows from noting that each quotient of eigenvalues is 
less than unity in absolute value, as a resuit of indexing the first eigenvalue as the 
dominant one, and therefore tends to zéro as that quotient is raised to successively 
higher powers. 

Thus, A^xo approaches a scalar multiple of vi. But any nonzero scalar multiple 
of an eigenvector is itself an eigenvector, so A^xo approaches an eigenvector of A 
corresponding to the dominant eigenvalue, providing c i is not zéro. The scalar c\ 
will be zéro only if xo is a linear combination of {V2, V3, . . . , \ n }- 

The power method begins with an initial vector xo, usually the vector having 
ail ones for its components, and then iteratively calculâtes the vectors 


xi = Ax 0 , 

X2 = Axi = A 2 xo, 

X 3 = AX2 = A 3 X0, 

Xk = Ax£_i = A^'xo. 


As k gets larger, x* approaches an eigenvector of A corresponding to its dominant 
eigenvalue. 

We can even détermine the dominant eigenvalue by scaling appropriately. If k 
is large enough so that Xk is a good approximation to the eigenvector, say to within 
acceptable roundoff error, then it follows from Eq. (1) that 


Ax k = X\x k . 


If Xk is scaled so that its largest component is unity, then the component of x* + i = 
Ax£ = X\Xk having the largest absolute value must be 

We can now formalize the power method. Begin with an initial guess xo for 
the eigenvector, having the property that its largest component in absolute value 
is unity. Iteratively, calculate xi, X2, X3, . . . by multiplying each successive iterate 
by A, the matrix of interest. Each time Xk(k = 1, 2, 3, . . .) is computed, identify 
its dominant component and divide each component by it. Redefine this scaled 
vector as the new x&. Each x* is an estimate of an eigenvector for A and each 
dominant component is an estimate for the associated eigenvalue. 


Example 1 Find the dominant eigenvalue, and a corresponding eigenvector for 



2 

3 


Solution We initialize xq = [1 l] T . Then 
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First Itération 



1 2 


1 


3 

xi = Ax 0 = 

4 3 


1 

— 

_7 J’ 


X & 7, 

X! 4- - [ 3 7] T = [0.428571 l] T . 

Second Itération 



1 2 


0.428571 


2.428571 

X 2 = Axi = 

4 3 


1 

— 

4.714286_|’ 


X « 4.714286, 


1 


x 2 


Third Itération 


[2.428571 4.714286] T = [0.515152 l] 


4.714286 


iT 



1 2 


0.515152 


2.515152 

X 3 = Ax 2 = 

4 3 


1 

— 

5.060606 ' 


À = 5.060606, 
1 


X3 


[2.515152 5.060606] 1 = [0.497006 1] 


5.060606 


iT 


Fourth Itération 


x 4 = Ax 3 = 

À « 4.988024, 
1 

x 4 < 


"1 2" 


"0.497006" 


"2.497006" 

4 3 


1 


4.988024 


[2.497006 4.988024] 7 = [0.500600 1] 


4.988024 


iT 


The method is converging to the eigenvalue 5 and its corresponding eigenvector 
[0.5 1] T . ■ 


Example 2 Find the dominant eigenvalue and a corresponding eigenvector for 



0 1 

0 0 

18 -1 
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Solution We initialize xo = [1 1 l] T . Then 

First Itération 


Xi = Axq = [1 1 10] T , 


X ~ 10, 

1 10 ] T 


[ 0.1 0.1 1 ] T . 


Second Itération 



" 0 

1 

ol 


"o.r 


" 0.1 " 

x 2 = Axi = 

0 

0 

1 


0.1 

— 

î 


18 

-1 

-7 


î 


-5.3 


X ^ -5.3, 


X2 


=b[ 


0.1 1 -5.3 


] 


T 


= -0.018868 -0.188679 



Third Itération 



"010 


”—0.018868 


"—0.188679" 

x 3 = Ax 2 = 

0 0 1 
18 -1 -7 


-0.188679 

1 

— 

1 

-7.150943 


À « -7.150943, 


x 3 


[-0.188679 1 — 7.150943l T 

— 7.150943 L J 

[0.026385 -0.139842 l] T . 


Continuing in this manner, we generate Table 6.1, where ail entries are rounded 
to four décimal places. The algorithm is converging to the eigenvalue —6.405125 
and its corresponding eigenvector 

[0.024376 -0.1561240 l] T . ■ 


Although effective when it converges, the power method has deficiencies. It 
does not converge to the dominant eigenvalue when that eigenvalue is complex, 
and it may not converge when there are more than one equally dominant eigen- 
values (See Problem 12). Furthermore, the method, in general, cannot be used to 
locate ail the eigenvalues. 

A more powerful numerical method is the inverse power method , which is the 
power method applied to the inverse of a matrix. This, of course, adds another 
assumption: the inverse must exist, or equivalently, the matrix must not hâve 
any zéro eigenvalues. Since a nonsingular matrix and its inverse share identical 




206 


Chapter 6 Eigenvalues and Eigenvectors 


Table 6.1 


Itération 

Eigen vector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

0.1000 

0.1000 

1.0000 

10.0000 

2 

-0.0189 

-0.1887 

1.0000 

-5.3000 

3 

0.0264 

-0.1398 

1.0000 

-7.1509 

4 

0.0219 

-0.1566 

1.0000 

-6.3852 

5 

0.0243 

-0.1551 

1.0000 

-6.4492 

6 

0.0242 

-0.1561 

1.0000 

-6.4078 

7 

0.0244 

-0.1560 

1.0000 

-6.4084 

8 

0.0244 

-0.1561 

1.0000 

-6.4056 


eigenvectors and reciprocal eigen- values (see Property 4 and Observation 1 of 
Section 6.4), once we know the eigenvalues and eigenvectors of the inverse of a 
matrix, we hâve the analogous information about the matrix itself. 

The power method applied to the inverse of a matrix A will generally converge 
to the dominant eigenvalue of A -1 . Its reciprocal will be the eigenvalue of A 
having the smallest absolute value. The advantages of the inverse power method 
are that it converges more rapidly than the power method, and if often can be used 
to find ail real eigenvalues of A; a disadvantage is that it deals with A -1 , which 
is laborious to calculate for matrices of large order. Such a calculation, however, 
can be avoided using LU décomposition. 

The power method generates the sequence of vectors 


x k = Axjc-i. 


The inverse power method will generate the sequence 

x k = A _1 x a -_i, 


which may be written as 


Ax k = x*_i. 


We solve for the unknown vector x k using LU-decomposition (see Section 3.5). 


Example 3 Use the inverse power method to find an eigenvalue for 




Solution We initialize xo = [1 1] T . The LU décomposition for A has A = LU 
with 




and U = 


2 1 
0 2 


First Itération. We solve the System LUxi = xo by first solving the System 
Ly = xq for y, and then solving the System Uxi = y for xi. Set y = [yi y2] J and 
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xi = [a b] T . The first System is 

yi +0^2 = 1, 

^1+^2 = 1, 

which has as its solution y\ = 1 and y 2 = 0. The System Uxi = y becomes 

2a + b = 1, 

2b = 0, 

which admits the solution a = 0.5 and b — 0. Thus, 
xi = A _1 xo = [0.5 0] T , 

À ~ 0.5 (an approximation to an eigenvalue for A -1 ), 

X! ^ T [0.5 0] T = [1 0] T . 

Second Itération. We solve the System LUx 2 = xi by first solving the System 
Ly = xi for y, and then solving the System Ux 2 = y for X 2 . Set y = [y\ y 2 ] J and 
X 2 = [a b] J . The first System is 

yi + 0y2 = 1 , 
yi + y2 = 0 , 

which has as its solution yi = 1 and y2 = — 1. The System Ux2 = y becomes 

2a + b — 1 , 

2/7= -1, 

which admits the solution a = 0.75 and b = —0.5. Thus, 

X2 = A _1 xi = [0.75 — 0.5] t , 

À « 0.75, 

X2 ^ _i_ [0.75 — 0.5 ] t = [1 — 0.666667] 1 . 

Third Itération. We first solve Ly = X2 to obtainy = [1 — 1.666667] 1 , and then 

UX3 = y to obtain X3 = [0.916667 — 0.833333] 1 . Then, 

À « 0.916667 

■t 

x 3 4- o 916667 [0.916667 — 0.833333] T = [l -0.909091] 1 . 

Continuing, we converge to the eigenvalue 1 for A -1 and its reciprocal 1/1 = 1 for 
A. The vector approximations are converging to [1 — l] T , which is an eigenvector 

for both A -1 and A. ■ 
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Example 4 Use the inverse power method to find an eigenvalue for 



7 2 0 
2 16 . 
0 6 7 


Solution We initialize xo = [1 1 l] T . The LU décomposition for A has 

A = LU with 



1 

0 

0 


"7 

2 

0“ 

L = 

0.285714 

1 

0 

and U = 

0 

0.428571 

6 


0 

14 

1 


0 

0 

-77 


First Itération 
Set y = [yi y 2 


y 3 ] T and xi = [a b c] 1 . The first System is 


T 


yi + 0 y 2 + 0y3 = 1, 


0.285714yi + y 2 + 0^3 = 1, 


Oyi + 14y2 + B = 1, 


which has as its solution y\ = 1, and y2 = 0.714286, and y 3 = —9. The System 
Uxi = y becomes 

la + 2b = 1, 

0.428571/7 + 6c = 0.714286, 

-77c = -9, 

which admits the solution a — 0.134199, b — 0.030303, and c = 0.116883. Thus, 

X! = A _1 x 0 = [0.134199 0.030303 0.116833] 1 , 

X « 0.134199 (an approximation to an eigenvalue for A -1 ), 

Xi [0.134199 0.030303 0.116833] 1 

i ✓ ,/ 

= [1 0.225806 0.870968] 1 . 
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Second Itération 

Solving the System Ly = xi for y, we obtain 

y = [1 -0.059908 1.709677] 1 . 

Then, solving the System Ux2 = y for X2, we get 

x 2 = [0.093981 0.171065 -0.022204] 1 ". 

Therefore, 

[0.093981 0.171065 -0.022204] 1 , 
1 — 0.129796] T . 

Third Itération 

Solving the System Ly = X2 for y, we obtain 

y = [0.549388 0.843032 -11.932245] 1 . 

Then, solving the System UX3 = y for X3, we get 

x 3 = [0.136319 -0.202424 0.154964] 1 . 


À » 0.171065, 
1 

' 2 ^ 0.171065 
= [0.549388 


Table 6.2 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

1.0000 

0.2258 

0.8710 

0.1342 

2 

0.5494 

1.0000 

-0.1298 

0.1711 

3 

-0.6734 

1.0000 

-0.7655 

-0.2024 

4 

-0.0404 

1.0000 

-0.5782 

-0.3921 

5 

-0.2677 

1.0000 

-0.5988 

-0.3197 

6 

-0.1723 

1.0000 

-0.6035 

-0.3372 

7 

-0.2116 

1.0000 

-0.5977 

-0.3323 

8 

-0.1951 

1.0000 

-0.6012 

-0.3336 

9 

-0.2021 

1.0000 

-0.5994 

-0.3333 

10 

-0.1991 

1.0000 

-0.6003 

-0.3334 

11 

-0.2004 

1.0000 

-0.5999 

-0.3333 

12 

-0.1998 

1.0000 

-0.6001 

-0.3333 
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Therefore, 


À « -0.202424, 
1 

3 -0.202424 

= [-0.673434 


0.136319 -0.202424 
1 — 0.765542] 1 . 


0.154964 


Continuing in this manner, we generate Table 6.2, where ail entries are rounded 
to four décimal places. The algorithm is converging to the eigenvalue —1/3 for 
A -1 and its reciprocal —3 for A. The vector approximations are converging to 
[—0.2 1 — 0.6] t , which is an eigenvector for both A -1 and A. ■ 


We can use Property 7 and Observation 4 of Section 6.4 in conjunction with 
the inverse power method to develop a procedure for finding ail eigenvalues and 
a set of corresponding eigenvectors for a matrix, providing that the eigenvalues 
are real and distinct, and estimâtes of their locations are known. The algorithm is 
known as the shifted inverse power method. 

If c is an estimate for an eigenvalue of A, then A — cl will hâve an eigenvalue 
near zéro, and its reciprocal will be the dominant eigenvalue of (A — cl) -1 . We 
use the inverse power method with an LU décomposition of A — cl to calculate 
the dominant eigenvalue À and its corresponding eigenvector x for (A — cl) -1 . 
Then 1 /À and x are an eigenvalue and eigenvector for A — cl while 1/À + c and x 
are an eigenvalue and eigenvector for A. 


Example 5 Find a second eigenvalue for the matrix given in Example 4. 


Table 6.3 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

0.6190 

0.7619 

1.0000 

-0.2917 

2 

0.4687 

0.7018 

1.0000 

-0.2639 

3 

0.3995 

0.6816 

1.0000 

-0.2557 

4 

0.3661 

0.6736 

1.0000 

-0.2526 

5 

0.3496 

0.6700 

1.0000 

-0.2513 

6 

0.3415 

0.6683 

1.0000 

-0.2506 

7 

0.3374 

0.6675 

1.0000 

-0.2503 

8 

0.3354 

0.6671 

1.0000 

-0.2502 

9 

0.3343 

0.6669 

1.0000 

-0.2501 

10 

0.3338 

0.6668 

1.0000 

-0.2500 

11 

0.3336 

0.6667 

1.0000 

-0.2500 
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Solution Since we do not hâve an estimate for any of the eigenvalues, we 
arbitrarily choose c — 15. Then 


A — cl — 



2 0 
-14 6 , 

6 -8 


which has an LU décomposition with 


1 

0 

0 


"-8 

2 

0 

0.25 

1 

0 

and U = 

0 

-13.5 

6 

0 

-0.444444 

1 


0 

0 

-5.333333 


Applying the inverse power method to A — 151, we generate Table 6.3, which 

is converging to À = —0.25 and x = | l] T . The corresponding eigenvalue 

of A is 1/ — 0.25 + 15 = 11, with the same eigenvector. 

Using the results of Examples 4 and 5, we hâve two eigenvalues, = —3 
and À. 2 = 11, of the 3x3 matrix defined in Example 4. Since the trace of a 
matrix equals the sum of the eigenvalues (Property 1 of Section 6.4), we know 
7 + 1 + 7 = — 3 + 11 + À3, so the last eigenvalue is À3 = 7. ■ 


Problems 6.6 


In Problems 1 through 10, use the power method to locate the dominant eigenvalue 
and a corresponding eigenvector for the given matrices. Stop after five itérations. 


1. 

'2 f 



2 . 

'2 

3" 


3 . 

"3 

6 


2 3_ 

•> 


_4 

6_ 

i 

_9 

6 

5 

4 . 

0 

1 


5 . 

"8 

2" 


6 . 

"8 

3' 


-4 ■ 

6_ 

i 

_3 

3 


_3 

2 

1 


"3 0 


0" 


"7 

2 

0" 


"3 

2 

3" 

7 . 

2 6 


4 

8 . 

2 

1 

6 

9 . 

2 

6 

6 


2 3 


5 


0 

6 

7 


3 

6 

11 



-17 7 

-4 1 . 

1 -14 


11. Use the power method on 



2 0 

2 2 

-1 0 



and explain why it does not converge to the dominant eigenvalue À = 3. 
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12 . Use the power method on 



and explain why it does not converge. 

13 . Shifting can also be used with the power method to locate the next most 
dominant eigenvalue, if it is real and distinct, once the dominant eigenvalue 
has been determined. Construct A — Àl, where À is the dominant eigenvalue 
of A, and apply the power method to the shifted matrix. If the algorithm 
converges to /x and x, then /x + X is an eigenvalue of A with the corresponding 
eigenvector x. Apply this shifted power method algorithm to the matrix in 
Problem 1. Use the results of Problem 1 to détermine the appropriate shift. 

14 . Use the shifted power method as described in Problem 13 to the matrix in 
Problem 9. Use the results of Problem 9 to détermine the appropriate shift. 

15 . Use the inverse power method on the matrix defined in Example 1. Stop after 
five itérations. 

16 . Use the inverse power method on the matrix defined in Problem 3. Take 

xo = [1 — 0.5] T and stop after five itérations. 

17 . Use the inverse power method on the matrix defined in Problem 5. Stop after 
five itérations. 

18 . Use the inverse power method on the matrix defined in Problem 6. Stop after 
five itérations. 

19 . Use the inverse power method on the matrix defined in Problem 9. Stop after 
five itérations. 

20 . Use the inverse power method on the matrix defined in Problem 10. Stop 
after five itérations. 

21 . Use the inverse power method on the matrix defined in Problem 11. Stop 
after five itérations. 

22. Use the inverse power method on the matrix defined in Problem 4. Explain 
the difficulty, and suggest a way to avoid it. 

23 . Use the inverse power method on the matrix defined in Problem 2. Explain 
the difficulty, and suggest a way to avoid it. 

24 . Can the power method converge to a dominant eigenvalue it that eigenvalue 
is not distinct? 

25. Apply the shifted inverse power method to the matrix defined in Problem 9, 
with a shift constant of 10. 

26 . Apply the shifted inverse power method to the matrix defined in Problem 10, 
with a shift constant of —25. 



Matrix Calculus 


7.1 Well-Defined Functions 

The student should be aware of the vast importance of polynomials and exponen- 
tials to calculus and differential équations. One should not be surprised to find, 
therefore, that polynomials and exponentials of matrices play an equally important 
rôle in matrix calculus and matrix differential équations. Since we will be inter- 
ested in using matrices to solve linear differential équations, we shall devote this 
entire chapter to defining matrix functions, specifically polynomials and exponen- 
tials, developing techniques for calculating these functions, and discussing some 
of their important properties. 

Let pk (x) dénoté an arbitrary polynomial in x of degree k , 

Pk(x) = a k x k + a k -\x k ~ l H h a\x + ao, (1) 

where the coefficients a k , a k ~ 1 , ...,a\,ao are real numbers. We then define 

Pki A) = cikA k + cik—i A k + • • • H - a \ A + üqI. (2) 

Recall from Chapter 1, that A 2 = A- A,A 3 = A 2 -A = A- A- A and, in general, 
A* = A* -1 • A. Also A 0 = I. 

Two observations are now immédiate. Whereas ao in (1) is actually multiplied 
by x° = 1, üq in (2) is multiplied by A 0 = I. Also, if A is an n x n matrix, then 
Pk{ A) is an n x n matrix since the right-hand side of (2) may be summed. 


Example 1 Find P 2 (A) for 



0 1 
0 0 
0 0 


0 

1 

0 


if P 2 (x) = 2x 2 + 3x + 4. 
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Solution In this case, p 2 (A) = 2 A 2 + 3 A + 41. Thus, 



"0 

1 

0 


"0 

1 

0 


1 

0 

0 " 

Pii A) = 2 

0 

0 

1 

+ 3 

0 

0 

1 

+ 4 

0 

1 

0 


0 

0 

0 


0 

0 

0 


0 

0 

1 



"0 

0 

1 


"0 

1 

0 


1 

0 

0 


“4 

3 

2" 

2 

0 

0 

0 

+ 3 

0 

0 

1 

+ 4 

0 

1 

0 

— 

0 

4 

3 


0 

0 

0 


0 

0 

0 


0 

0 

1 


0 

0 

4 


Note that had we defined /? 2 (A) = 2A 2 + 3 A + 4 (that is, without the I term), 
we could not hâve performed the addition since addition of a matrix and a scalar 
is undefined. ■ 


Since a matrix commutes with itself, many of the properties of polynomials 
(addition, subtraction, multiplication, and factoring but not division) are still valid 
for polynomials of a matrix. For instance, if f(x ) , d(x), q(x), and r(x) represent 
polynomials in x and if 


f(x) = d(x)q(x) + r(x) (3) 

then it must be the case that 

/(A) = d(A)q(A) + r(A). (4) 

Equation (4) follows from (3) only because A commutes with itself; thus, we 
multiply together two polynomials in A precisely in the same manner that we 
multiply together two polynomials in x. 

If we recall from calculus that many functions can be written as a Maclaurin 
sériés, then we can define functions of matrices quite easily. For instance, the 
Maclaurin sériés for e x is 



Thus, we define the exponential of a matrix A as 




The question of convergence now arises. For an infinité sériés of matrices we 
define convergence as follows: 

Définition 1 A sequence {B&} of matrices, = [&*•], is said to converge to a 

«/ 

matrix B = [btj] if the éléments b\- converge to b\j for every i and j. 
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Définition 2 The infinité sériés J2T=o B, n , converges to B if the sequence {S&} of 
partial sums, where S* = J2n=o B, x , converges to B. 

It can be shown (see Theorem 1, this section) that the infinité sériés given in 
(6) converges for any matrix A. Thus e A is defined for every matrix. 


Example 2 Find e A if 


Solution 


A = 


2 0 ' 
0 0 


A 

e = 


'2 0 ' 
0 0 


1 0' 
0 1 


1 


2 0 ' 
0 0 


1 

+ 2 ! 


2 0 ' 
0 0 


H 2 


1 

+ 3! 


'2 0 
0 0 


1 0' 
0 1 


+ 


' 2 / 1 ! 0 ' 
0 0 


+ 


oo 


£ 2 k /k! 0 

k=0 

0 1 


, 2 

0 


2 2 /2! 0 

. 0 0J 


0' 

.0 


+ 


’2 3 /3! 0' 
0 0 


+ 


-|3 


+ 


In general, if A is the diagonal matrix 

0 
0 


then we can show (see Problem 12) that 

IV 1 0 


L 0 0 

If A is not a diagonal matrix, then it is very difficult to find e A directly from the 
définition given in (6). For an arbitrary A, e A does not hâve the form exhibited in 
(7). For example, if 



A = 


Ài 0 

0 A. 2 

0 0 


1 2 
4 3 


A = 


? 
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it can be shown (however, not yet by us) that 

A 1 [2e 5 -h 4e- 1 2e 5 - 2e~ x ~ 

6 4e 5 — 4e ~ 1 4e 5 + 2e~ l 

For the purposes of this book, the exponential is the only function that is 
needed. However, it may be of some value to know how other functions of matrices, 
sines, cosines, etc., are defined. The folio wing theorem, the proof of which is beyond 
the scope of this book, provides this information. 

Theorem 1 Let z represent the complex variable x + iy. If f(z ) has the Taylor 
sériés YlkLo a kZ k , which converges for \z\ < R, and if the eigenvalues k\, À 2 , . . . , X n 
of an n x n matrix A hâve the property that |À/| < R(/ = 1, 2, . . . , n), then 
akA k will converge to an n x n matrix which is defined to be /(A). In such a 
case, /(A) is said to be well defined. 


Example 3 DefinesinA. 

Solution A Taylor sériés for sin z is 


oo 


sinz 


= £ 


k = 0 


(-1 ) k z 2k+1 
(2k + 1)! 


3 5 7 

Z Z Z 

— z — — + — — — 

3! 5! 7! 


This sériés can be shown to converge for ail z (that is, R — oo). Hence, since 
any eigenvalue À of A must hâve the property |À| < oo (that is, À is finite) sin A 
can be defined for every A as 


sin 


oo 


a = £ 

k=0 


(~l) k A 2k+1 
(2k + l)\ 





Problems 7.1 


1. Let q(x) = x — 1. Find p^i A) and q(A)pk(A) if 


(a) 

(b) 



2 

-1 

0 

2 " 

4 ’ 


3 

4 , 
1 


k — 2, and p 2 (v) = jv 2 — 2jc + 1, 


k — 3, and P 3 (x) = 2x 3 — 3x 2 + 4. 
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2. If pk(x) is defined by (1), find pk( A) for the diagonal matrix 



0 0 
À2 0 . 

0 À 3 


Can you generalize? 


3. By actually computing both sides of the following équation separately, verify 
that (A - 31) (A + 21) = A 2 - A - 61 for 



"1 2" 


1 0-2" 

(a) A = 

_3 4_ 

(b) A = 

3 1 1 

-2 -2 3 


The above équation is an example of matrix factoring. 

4. Although x 2 — y 2 = (x — y)(x + y) whenever x and y dénoté real-valued 
variables, show by example that A 2 — B 2 need not equal the product 
(A — B) (A + B) whenever A and B dénoté 2x2 real matrices. Why? 

5. It is known that x 2 — 5x + 6 factors into the product (x — 2)(x — 3) whenever 
x dénotés a real-valued variable. Is it necessarily true that A 2 — 5A + 61 = 
(A — 21) (A — 31) whenever A represents a square real matrix? Why? 

6. Détermine lim^oo B^ when 






(0.5)* 


7. Détermine lim / ' _ >oc B^ when 


2k 

k TI 



k + 3 

k 2 - 2k + 1 


3 A: 2 + 2k 
2k 2 


8. Détermine lim^oo when 



( 0 . 2 )* 1 ( 0 . 1 )* 
4 3* 0 
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9. It is known that arctan (z) = l) w /(2ft + 1 )]z 2/J+1 converges for ail 

|z| < 7r/2. Détermine for which of the folio wing matrices A, arctan(A) = 
ESloK- 1 )"/ 211 + 1]A 2 " +1 is well defined: 




"0 1 0" 


"i 2 r 


o 

O 
1 

(d) 

0 0-1 
_0 1 0_ 

(e) 

0 3 5 

_0 -1 — 3 _ 

(f) 

0 0 1 

0—12 
LU 8 4 J 


10 . It is known that ln(l + z) = l) ,î+1 /«]z” converges for ail |z| < 1. 

Détermine for which of the matrices given in Problem 9 ln(I + A) = 

l) /7+1 / 77 ]A 77 is well defined. 

11. It is known that f(z ) = z n /3” converges for ail \z\ < 3. Détermine for 
which of the matrices given in Problem 9 /(A) = A n /3 n is well defined. 

12 . Dérivé Eq. (7). 

13 . Find e A when 



0" 

2 


14 . Find e A when 



0 " 

28 ‘ 


15 . Find e A when 



0 " 

0 . 
0 


16 . Dérivé an expression for sin(A) similar to Eq. (7) when A is a square diagonal 
matrix. 

17 . Find sin(A) for the matrix given in Problem 13. 

18 . Find sin(A) for the matrix given in Problem 14. 
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19. Using Theorem 1, give a définition for cos A and use this définition to find 

"1 0 " 
cos |^0 2 • 

20. Find cos(A) for the matrix given in Problem 15. 

7.2 Cayley-Hamilton Theorem 

We now State one of the most powerful theorems of matrix theory, the proof which 
is given in the Final Comments at the end of this chapter. 

Cayley-Hamilton Theorem. A matrix satisfies its own characteristic équation. That 
is, if the characteristic équation ofan n x n matrix A is X n + a n -\X n ~ 1 + • • • + a\k + 
ao = 0, then 

A n T a n —\ A. 11 1 T • • • T a\ A T afl — 0. 

Note once again that when we change a scalar équation to a matrix équation, 
the unity element 1 is replaced by the identity matrix I. 

Example 1 Verify the Cayley-Hamilton theorem for 



Solution The characteristic équation for A is À 2 — 4k — 5 = 0. 



Example 2 Verify the Cayley-Hamilton theorem for 


3 0-1 
A = 2 0 1 

0 0 4 
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Solution The characteristic équation of A is (3 — À) (—À) (4 — À) = 0. 


(31 — A) (—A) (41 — A) 


"3 

0 

i 

o 


"3 

0 

-1" 

0 

3 

0 

— 

2 

0 

1 

1 o 

0 

3 


0 

0 

4 

"4 

0 

0" 


"3 

0 

-1“ 

0 

4 

0 

— 

2 

0 

1 


0 

4 


0 

0 

4 




i 

o 

o 

h- * 

1 


-3 0 1 

— 

-2 3 -1 


-2 0 -1 


0 0 -1 


0 0-4 


"00 1“ 


"-3 0 -3“ 

— 

-2 3 -1 


-2 0 -2 


0 0-1 


0 0 0 


1 0 1 
-2 4 -1 
0 0 0 



0 0 
0 0 
0 0 


= 0 . ■ 


One immédiate conséquence of the Cayley-Hamilton theorem is a new 
method for finding the inverse of a nonsingular matrix. If 


X n T a n —\X n T • • • T a\X T ÜQ — 0 


is the characteristic équation of a matrix A, it follows from Problem 17 of Section 
6.4 that det(A) = (— l) n aQ. Thus, A is invertible if and only if ao ^ 0. 

Now assume that ao ^ 0. By the Cayley-Hamilton theorem, we hâve 

A" T ci n ~\ A /z l T * * * T a\ A T ciq\ — 0, 

A[ A ,z 1 + ci n ~ i A ,? 2 + • • • + #il] = —ciqI, 


or 



(A /z ^ + a n — iA” 2 + • • • + ûql) 

ao 



Thus, (— l/flo)(A" 1 + a n - iA” 2 + • • • + ail) is an inverse of A. But since the 
inverse is unique (see Theorem 2 of Section 3.4), we hâve that 

A 1 = — (A ,z i 1 A” 2 + • • • + a\l). (9) 

ao 


Example 3 Using the Cayley-Hamilton theorem, find A 1 for 

1 -2 4“ 

0 - 12 . 

2 0 3 


A = 
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Solution The characteristic équation for A is A 3 — 3A 2 — 9X + 3 = 0. Thus, by 
the Cayley-Hamilton theorem, 

A 3 - 3A 2 - 9A + 31 = 0. 


Hence 

A 3 - 3A 2 - 9A = -31, 
A (A 2 - 3A - 91) = -31, 


or, 


Thus, 


A -1 = I - 




-A 2 + 3A + 9l) 



f-9 0 -12 


"3 -6 12” 


1 

O 

O 
1 

\ 

1 

r-H 

1 

1 

+ 

0-3 6 

+ 

0 9 0 


1 

t-H 

1 

00 

1 

1 


6 0 9 


i 

o 

O 

V» O 

1 

) 


1 

3 


'3 -6 0' 
-4 5 2 

-2 4 1 


Problems 7.2 

Verify the Cayley-Hamilton theorem and use it to find A -1 , where possible, for: 


1. A 

"1 

3 

2" 

4 

5 

2. A = 

"1 

2 

2" 

4 y 



"2 

0 

1" 


"1 

-1 

2" 

3. A = 

4 

0 

2 

, 4. A = 

0 

3 

2 


0 

0 

-1 


2 

1 

2 


10 0 0 
0-1 0 0 
0 0 - 10 * 
0 0 0 1 


5. A = 
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7.3 Polynomials of Matrices — Distinct Eigenvalues 

In general, it is very difficult to compute functions of matrices from their définition 
as infinité sériés (one exception is the diagonal matrix). The Cayley-Hamilton 
theorem, however, provides a starting point for the development of an alternate, 
straightforward method for calculating these functions. In this section, we shall 
develop the method for polynomials of matrices having distinct eigenvalues. In 
the ensuing sections, we shall extend the method to functions of matrices having 
arbitrary eigenvalues. 

Let A represent an n x n matrix. Define d(k) — det(A — AI). Thus, d(k) is 
an nt\i degree polynomial in k and the characteristic équation of A is d(k) — 0. 
From Chapter 6, we know that if À; is an eigenvalue of A, then À, is a root of the 
characteristic équation, hence 

d(ki) = 0. (10) 

From the Cayley-Hamilton theorem, we know that a matrix must satisfy its own 
characteristic équation, hence 

d( A) = 0. (11) 

Let /(A) be any matrix polynomial of arbitrary degree that we wish to compute. 
/(À) represents the corresponding polynomial of À. A theorem of algebra States 
that there exist polynomials q(k) and r(À) such that 

f(k) = d(k)q(k) + r(k), (12) 

where r(À) is called the remainder. The degree of r(k) is less than that of d(k), 

which is n , and must be less than or equal to the degree of f(k) (why?). 


Example 1 Find q(k) and r(À) if f(k) = À 4 + 2 À 3 — 1 and d(k) = k 2 — 1. 


Solution For k ^ ±1, d{k) ^ 0. Dividing f(k) by d(k ), we obtain 


m 

d{k) 

m 

d(k) 


k 4 + 2à 3 - 1 
À 2 -l 


^7 ? + 2k + 1^ 




+ 2k + 1^ 


+ 


2k 

d(Xj' 


or 


f(k) = d(k) (k 2 + 2k + l) (2k). (13) 

If we define q(k) — k 2 + 2k + 1 and r(À) = 2k, (13) has the exact form of (12) for 
ail À except possibly k = ±1. However, by direct substitution, we find that (13) is 
also valid for k = ±1; hence (13) is an identity for ail (À). ■ 
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From (12), (3), and (4), we hâve 

/(A) = d(A)q(A) + r(A). (14) 

Using (11), we obtain 

/(A) = r(A). (15) 

Therefore, it follows that any polynomial in A may be written as a polynomial of 
degree n — 1 or less. For example, if A is a 4 x 4 matrix and if we wish to compute 
/(A) = A 957 — 3 A 59 + 2 A 3 — 41, then (15) implies that /(A) can be written as a 
polynomial of degree three or less in A, that is, 

A 957 - 3A 59 + 2A 3 - 41 = a 3 A 3 + <* 2 A 2 + «iA + aol (16) 

where « 3 , « 2 , ai, ao are scalars that still must be determined. Once « 3 , « 2 , 04 , ao 
are computed, the student should observe that it is much easier to calculate the 
right side rather than the left side of (16). 

If A is an n x n matrix, then r(A.) will be a polynomial having the form 

r(À) — oi n —\X n 3 + ct n — 2^ n 2 H - * * * oqA, + «o- (17) 

If Xi is an eigenvalue of A, then we hâve, after substituting (10) into (12), that 

f(Xi) = r(Xi). (18) 

Thus, using (17), Eq. (18) may be rewritten as 

/U«) — a n-l(hi) n 1 + 2 + * * * + Ot\ (À/) + «0 (19) 

if À,* is an eigenvalue. 

If we now assume that A has distinct eigenvalues, X \ , X 2 , . . . , X n (note that if the 
eigenvalues are distinct, there must be n of them), then (19) may be used to gen- 
erate n simultaneous linear équations for the n unknowns a n - 1 , ot n - 2 , . . . , oq, «o: 

/U 1 ) = KM) = af w _i(A.i) /î_1 + a n - 2 (M) H_2 H h ai (A 4 ) + ûfo, 

/(A. 2) = KM) — û' /î _i(À 2 ) /î 1 + CL n - 2(^2)" 2 + * * * + û?i(À 2 ) + û'o, 

/(M) — KM) ~ a n-l(^n) n 1 + û? n _2(A. w )” 2 + * * * + Ot\(k n ) + «O- (20) 

Note that /(A.) and the eigenvalues A 4 , A. 2 , . . . , X n are assumed known; hence 

/(A 4 ), /(A. 2 ), • . . , f(X n ) are known, and the only unknowns in (20) are a n -\, 
oi n - 2 , •••,«!, «q. 
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Example 2 Find A 593 if 



Solution The eigenvalues of A are X\ = 1, À 2 = — 1. For this example, /(A) = 
A 593 , thus, /(À) = À 593 . Since A is a 2 x 2 matrix, r(A) will be a polynomial of 
degree (2 — 1) = 1 or less, hence r(A) = ai A + aol and r(À) = ot\k + ao. From 
(15), we hâve that /(A) = r(A), thus, for this example, 


A 593 = ai A + agi. 



From (18), we hâve that /(À,-) = r(À z ) if À*- is an eigenvalue of A; thus, for this 
example, (À/) 593 = aiÀ z + ao. Substituting the eigenvalues of A into this équation, 
we obtain the following System for ai and ao. 

(I) 593 = ai(l) + ar 0 , 

(-1) 593 = ai(-l) + a 0 , 


or 


1 = ai + a 0 , 

— 1 = —ai + ao. (22) 

Solving (22), we obtain ao = 0, ai = 1. Substituting these values into (21), we 
obtain A 593 = 1 • A + 0 • I or 


"-3 

-4" 

593 

"-3 

-4" 

2 

3 


2 

3 



Example 3 Find A 39 if 




Solution The eigenvalues of A are = 5, X 2 = 2. For this example, /(A) = A 39 , 
thus f(k) = À 39 . Since A is a 2 x 2 matrix, r(A) will be a polynomial of degree 1 
or less, hence r(A) = ai A + aol and r(À.) = aiÀ + ao. From (15), we hâve that 
/(A) = r(A), thus, for this example, 

A 39 = ai A + agi. 


(23) 



7.3 Polynomials of Matrices — Distinct Eigenvalues 


From (18) we hâve that /(À/) = r(À/) if À/ is an eigenvalue of A, thus for this 
example, (À/) 39 = ol\ À/ + «o- Substituting the eigenvalues of A into this équation, 
we obtain the following System for ot\ and ao: 

_39 _ 

J = Jûq + û'o, 

2 39 = 2a i + «„• (24) 


Solving (24), we obtain 


539 _ 2 39 

ai = , «0 


— 2(5) 39 + 5(2) 


39 


Substituting these values into (23), we obtain 


A 39 = 


5 39 - 2 39 

"4 1" 

— 2(5) 39 + 5(2) 39 

"1 0" 

3 

2 3 

+ 3 

0 1 


1 

3 


2(5) 39 + 2 39 5 39 - 2 39 

2(5) 39 - 2(2) 39 5 39 + 2(2) 39 


(25) 


The number 5 39 and 2 39 can be determined on a calculator. For our purposes, 
however, the form of (25) is sufficient and no further simplification is required. ■ 


Example 4 Find A 602 - 3 A 3 if 





Solution The eigenvalues of A are — 0, À 2 = 1, À 3 = 3. 

/(A) = A 602 - 3A 3 , r(A) = a 2 A 2 + a\A + a 0 I, 
/(À) = À 602 — 3 à 3 , r(X) = CL2 h 2 + + û'o- 


Note that since A is a 3 x 3 matrix, r(A) must be no more than a second degree 
polynomial. Now 


/(A) = r(A); 


thus, 


A 602 - 3 A 3 = a 2 A 2 + a, A + «(,!. 


(26) 
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If Xi is an eigenvalue of A, then f(Xi) — r(À/). Thus, 

(À,) 602 - 3(À,) 3 = a 2 i}-i) 2 + ai A.,- + a 0 ; 

hence, 

(0) 602 - 3 (O) 3 = a 2 (0) 2 + a, (0) + a 0 , 

(1) 602 _ 3 (1) 3 = a2(1) 2 + ai(1) + ao> 
(3)602 _ 3(3)3 = 0 , 2 (3)2 + ai(3) + aQï 


or 


0 = ao, 

-2 = Œ2 + a\ + (xq, 


3 602 - 81 = 9cx 2 + 3ai + a 0 . 


Thus, 


3 602 - 75 


Ci2 = 


— (3) 602 + 63 

«1 = — , (XQ = ü. 


Substituting (27) into (26), we obtain 

^602 7c 
A 602 _ 3A 3 = J - 7 ^ 


1 

6 


'-12 6(3) 602 — 498 -6(3) 602 + 474' 

0 0 0 

0 — 6(3) 602 + 486 6(3) 602 - 486 


(27) 


"1 

10 

-8 


— (3) 602 + 63 

Z 

"1 

4 

-2" 

0 

0 

0 

+ 

0 

0 

0 

0 

-9 

9 


6 

0 

-3 

3 


Finally, the student should note that if the polynomial to be calculated is already 
of a degree less than or equal to n — 1, then this method affords no simplification 
and the polynomial must still be computed directly. 


Problems 7.3 


1. Specialize System (20) for /(A) = A 7 and 



3 

2 


Solve this System and use the results to détermine A 7 . Check your answer by 
direct calculations. 



7.3 Polynomials of Matrices — Distinct Eigenvalues 


2. Find A 50 for the matrix A given in Problem 1. 

3. Specialize System (20) for /(A) = A 735 and 


A = 


0 1 
0 -1 


Solve this System and use the results to détermine A 735 ( What do you notice 
about A 3 ?). 

4. Specialize System (20) for /(A) = A 20 and 


A = 


-3 6 
-1 2 


Solve this System and use the results to détermine A 20 . 

5. Find A 97 for the matrix A given in Problem 4. 

6. Find A 50 for the matrix A given in Example 3. 

7. Specialize System (20) for /(A) = A 78 and 




Solve this System and use the results to détermine 


8 . Find A 41 for the matrix A given in Problem 7. 

9 . Specialize System (20) for /(A) = 

A 222 and 


"1 -1 2" 

A = 

0-12 


0 0 2 



Solve this System and use the results to détermine A 222 . 

10 . Specialize System (20) for /(A) = A 17 , when A is a 3 x 3 matrix having 3, 5, 
and 10 as its eigenvalues. 

11. Specialize System (20) for /(A) = A 25 , when A is a 4 x 4 matrix having 
2, —2, 3, and 4 as its eigenvalues. 

12 . Specialize System (20) for /(A) = A 25 , when A is a 4 x 4 matrix having 
1, —2, 3, and —4 as its eigenvalues. 

13 . Specialize System (20) for /(A) = A 8 , when A is a 5 x 5 matrix having 
1, —1, 2, —2, and 3 as its eigenvalues. 

14 . Specialize System (20) for /(A) = A 8 — 3 A 5 + 51, when A is the matrix 
described in Problem 10. 
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15. Specialize System (20) for /(A) = A 8 — 3 A 5 + 51, when A is the matrix 
described in Problem 11. 

16. Specialize System (20) for /(A) = A 8 — 3 A 5 + 51, when A is the matrix 
described in Problem 12. 

17. Specialize System (20) for /(A) = A 10 + 6A 3 + 8A, when A is the matrix 
described in Problem 12. 

18. Specialize System (20) for /(A) = A 10 + 6A 3 + 8A, when A is the matrix 
described in Problem 13. 

19. Find A 202 - 3 A 147 + 21 for the A of Problem 1. 

20. Find A 1025 — 4A 5 for the A of Problem 1. 

21. Find A 8 — 3A 5 — I for the matrix given in Problem 7. 

22. Find A 13 - 12A 9 + 51 for 




23. Find A 10 — 2A 5 + 101 for the matrix given in Problem 22. 

24. Find A 593 — 2A 15 for 



-2 4 
0 0 
-1 5 


3 

0 . 
2 


25. Specialize System (20) for /(A) = A 12 — 3 A 9 + 2 A + 51 and 



0 1 
0 0 
-4 4 


0 

1 

1 


Solve this System, and use the results to détermine /(A). 

26. Specialize System (20) for /(A) = A 9 — 3A 4 + I and 



’ 0 1 

0 0 

2_ I 

L 16 4 



Solve this System, and use the results to détermine /(A). 


7.4 Polyriomials of Matrices — General Case 


The only restriction in the previous section was that the eigenvalues of A had 
to be distinct. The following theorem suggests how to obtain n équations for the 
unknown a’s in (15) even if some of the eigenvalues are identical. 



7.4 Polynomials of Matrices — General Case 


Theorem 1 Let /(A.) and r(X) be defined as in Eq. (12). If Xi is an eigenvalue of 
multiplicity k, then 


fO-i) = r(Xi), 

4/U ,) _ dr(Xj) 
dX dX 

d 2 f(Xj) = d 2 r(Xj) 
dX 2 dX 2 



d k ~ l AXj) = d^HXj) 
dX k ~ l dX k ~ l 


where the notation d n f(Xi)/dX n dénotés the nth dérivative of f(X) with respect to 
X evaluated at X = À/.* 

Thus, for example, if Xi is an eigenvalue of multiplicity 3, Theorem 1 implies 
that f(X) and its first two dérivatives evaluated at X = Xi are equal, respectively, to 
r(X) and its first two dérivatives also evaluated at X = Xi. If À; is an eigenvalue of 
multiplicity 5, then /(À) and the first four dérivatives of f(X) evaluated at X = Xi are 
equal respectively to r(À) and the first four dérivatives of r(X) evaluated at X = À/. 
Note, furthermore, that if À/ is an eigenvalue of multiplicity 1, then Theorem 1 
implies that /(À/) = r(À/), which is Eq. (18). 


Example 1 Find A 24 - 3 A 15 if 




Solution The eigenvalues of A are 
eigenvalue of multiplicity three. 

/(A) = A 24 - 3A 15 
f(X) = X 24 - 3À 15 
f'{X) = 24A 23 - 45À 14 
f"{X) = 552À 22 - 630A 13 


Xi = X 2 = X 3 = 1; hence, X = 1 is an 

r( A) = (X 2 A 2 + ot\ A + aol 
r(X) — OL 2 X 2 + ol\X + aç) 
r'(X) = 20 L 2 X + a\ 
r"(X) = 2a 2 . 


* 


Theorem 1 is proved by differentiating Eq. (12) k — 1 times and noting that if À/ is an eigenvalue of 
multiplicity k, then 


d{Xi) 


d[d(ki )] 


A k - l U(Xj) 

dX k ~ l 



dX 
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Now /(A) = r( A), hence 

A 24 - 3A 15 = a 2 A 2 + «iA + a 0 I. (29) 

Also, since X = 1 is an eigenvalue of multiplicity 3, it folio ws from Theorem 1 that 

/( 1) = KD, 
fa) = r"( D. 


Hence, 


or 


(l) 24 - 3(1) 15 = a 2 (l) 2 + ai(l) + ao, 
24(1) 23 — 45(1) 14 = 2a 2 (l) + ai, 
552(1) 22 — 630(1) 13 = 2a 2 , 

—2 = «2 + û?l + ^0, 

—21 = 2û? 2 + ai, 

—78 = 2a?2* 


Thus, «2 = —39, a\ — 57, cxq = —20, and from Eq. (29) 


A 24 - 3A 15 = —39 A 2 + 57A - 201 = 


-44 

270 

-84 

0 

-2 

0 

21 

-93 

40 


Example 2 Set up the necessary équation to find A 15 — 6A 2 if 

"1 4 3 2 1 -7' 

0 0 2 11 1 0 

0 0 1-1 0 1 

A_ 0 0 0 -1 2 1 * 

0 0 0 0 -1 17 

0 0 0 0 0 1 

Solution The eigenvalues of A are X\ = X^ = À3 = 1, X^ — X^ — —1, X^ — 0. 


/(A) = A 15 - 6A 2 
/(À) = À 15 - 6X 2 
f'(X) = 15X 14 - 12X 
f”( X) = 210À 13 - 12 


r( A) = asA 5 + a4A 4 + a3A 3 + ai A 2 + a\ A + aol 
r(À) = + 014 X + « 3 À + + oc\X + ao 

/(T-) = 5a 5 À 4 + 4û(4À 3 + 3of3À 2 + 2a2X + oq 
r"(À) = 20 û' 5 À 3 4- 12 û( 4 À 2 + 6 a^X + 2 ^ 2 . 
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Since /(A) = r( A), 

A 15 — 6A 2 = a$A 5 + CK4A 4 + o^A 3 + CK2A 2 + a\A + ckoI. (30) 

Since X = 1 is an eigen value of multiplicity 3, X = — 1 is an eigenvalue of multi- 
plicity 2 and X = 0 is an eigenvalue of multiplicity 1, it follows from Theorem 1 
that 

/(l) = r(l), 

/(l) = r'd), 

/"(O = r"(l), (31) 

/(-l) = r(-l), 

/'(-l) = r'(-l), 

/(O) = r( 0 ). 

Hence, 

(l ) 15 - 6 ( 1) 2 = a 5 (l ) 5 + a 4 (l ) 4 + a 3 (l ) 3 + a 2 (l ) 2 + «i(l) + «o 
15(1) 14 - 12(1) = 5a 5 (l ) 4 + 4a 4 (l ) 3 + 3a 3 (l ) 2 + 2a 2 (l) + a\ 

210(1) 13 - 12 = 20o , 5 (1) 3 + 12a 4 (l) 2 + 6a 3 (l) + 2» 2 
(-1) 15 - 6(— l) 2 = asi-l) 5 + »4(-l) 4 + «3(-l) 3 + a2(-l) 2 + ai(-l) + «o 
15(— l) 14 - 12(— 1) = 5a 5 (— l ) 4 + 4a 4 (— l ) 3 + 3a 3 (-l ) 2 + 2a 2 (-l) + a, 

(O) 15 - 12(0) 2 = a 5 (0) 5 + a 4 (0) 4 + a 3 (0) 3 + a 2 (0) 2 + ai(0) + ao 

or 

-5 = as + (X 4 + «3 + OL 2 + ai + ao 
3 = 5a 5 + 4af4 + 3a$ + 2a2 4- o?i 
198 = 20 û( 5 + 12af4 + 6ûf3 + 2a2 (32) 

—7 = —CK 5 + CK4 — CK3 + CK2 — CKl + CKO 

27 = 5ck5 — 4ck4 + 3ck3 — 2a2 + o?i 
0 = ckq. 


System (32) can now be solved uniquely for a 5, «4 , . . . , «o; the results are then 
substituted into (30) to obtain /(A). ■ 
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Problems 7.4 

1. Using Theorem 1, establish the équations that are needed to find A 7 if A is a 
2x2 matrix having 2 and 2 as multiple eigenvalues. 

2. Using Theorem 1, establish the équations that are needed to find A 7 if A is a 
3x3 matrix having 2 as an eigenvalue of multiplicity three. 

3. Redo Problem 2 if instead the eigenvalues are 2, 2, and 1. 

4. Using Theorem 1, establish the équations that are needed to find A 10 if A is 
a 2 x 2 matrix having 3 as an eigenvalue of multiplicity two. 

5. Redo Problem 4 if instead the matrix has order 3x3 with 3 as an eigenvalue 
of multiplicity three. 

6. Redo Problem 4 if instead the matrix has order 4x4 with 3 as an eigenvalue 
of multiplicity four. 

7. Using Theorem 1, establish the équations that are needed to find A 9 if A is a 
4x4 matrix having 2 as an eigenvalue of multiplicity four. 

8. Redo Problem 7 if instead the eigenvalues are 2, 2, 2, and 1. 

9. Redo Problem 7 if instead the eigenvalues are 2 and 1, both with multiplicity 
two. 

10. Set up (but do not solve) the necessary équations to find A 10 — 3 A 5 if 

-T 
1 

-3 
2 * 

0 
5 


(First find A 6 using Theorem 1, and then by direct multiplication.) 
12. Find A 521 if 


A = 


'5 

0 

0 

0 

0 

0 


■2 

5 

0 


1 

2 

5 


0 0 
0 0 
0 0 


1 

1 

0 

2 

0 

0 


5 

-1 

1 

1 

2 

0 


11. Find A 6 in two different ways if 


A = 


5 8' 

-2 -5 
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13 . Find A 14 - 3A 13 if 



4 1 
0 0 
-8 1 



7.5 Functions of a Matrix 

Once the student understands how to compute polynomials of a matrix, com- 
puting exponentials and other functions of a matrix is easy, because the methods 
developed in the previous two sections remain valid for more general functions. 

Let f(X) represent a function of X and suppose we wish to compute /(A). It 
can be shown, for a large class of problems, that there exists a function q(X ) and 
an n — 1 degree polynomial r(À) (we assume A is of order n x n) such that 

f(X) = q(X)d(X) + r(X), (33) 

where d(X) — det(A — AI). Hence, it follows that 

/(A) = q(A)d(A) + r(A). (34) 

Since (33) and (34) are exactly Eqs. (12) and (14), where /(À) is now understood 
to be a general function and not restricted to polynomials, the analysis of Sections 
7.3 and 7.4 can again be applied. It then follows that 

(a) /(A) = r(A), and 

(b) Theorem 1 of Section 7.4 remains valid 

Thus, the methods used to compute a polynomial of a matrix can be generalized 
and used to compute arbitrary functions of a matrix. 


Example 1 Find e A if 



2 

3 


Solution The eigenvalues of A are Ài = 5, X 2 = — 1; thus, 

/(A) = e A r(A) = a\ A + aol 

f(X)=e k r(X) — Œ{X œq. 


Now /(A) = r(A); hence 



ai A + a 0 I. 


(35) 
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Also, since Theorem 1 of Section 7.4 is still valid, 

/(5) = r(5), 


and 

/(-l) = r(-l); 

hence, 

e 5 = 5o?i + «o, 
e -1 = — ai + û'o- 

Thus, 


a\ 



and 


ao = 


e 5 + 5e 1 
6 


Substituting these values into (35), we obtain 



1 2e 5 4- 4e 1 
6 4e 5 — 4e _1 


2e 5 — 2e 
4e 5 4- 2e 



Example 2 Find e A if 



1 0 

2 1 . 

0 2 


Solution The eigenvalues of A are X\ — X 2 — 7-3 = 2, thus, 


/(A) = e A 

/(^) = e ; - 
/'(*) = e x 

fw = e À 


r(A) = «2 A 2 + ai A + aol 
r(À) = a2 ^ 2 + ai A. + ao 
r'(X) = 2a 2 X 4 - oi\ 
r"(k) — 2a 2 . 


since /(A) = r(A), 


e — a 2 A 4- A + aol. 


(36) 



7.5 Functions of a Matrix 


Since À = 2 is an eigenvalue of multiplicity three, 

/( 2) = r(2). 

f ( 2 ) = /( 2 ), 

/"(2) = r" (2); 


hence. 


O 

£ = 4a 2 + 2ai + û?o, 
£ = 4a 2 + ot \ , 

£ 2 = 2a 2 , 


or 



Substituting these values into (36), we obtain 


p 2 

"4 

4 

1 


"2 

1 

0 " 


"1 

0 

0 


V 

e 2 e 2 /2" 

e A = y 

0 

4 

4 

-, 2 

0 

2 

1 

+, 2 

0 

1 

0 

— 

0 

e 2 e 2 

2 

0 

0 

4 


0 

0 

2 


0 

0 

1 


_0 

1 

(N 

O 


Example 3 FindsinAif 



1 

71 

1 


0 

0 

7t/2 


Solution The eigenvalues of A are X\ — 7i/2, à 2 = X$ = 7r; thus 


/(A) = sin A 
f(X) = sin À 
f'(X) = cos À 


r(A) = a 2 A 2 + ai A + aol 
r(X) = a 2 À 2 + ot\X + ao 
r'(X) = 2a 2 À + ai. 


But /(A) = r( A), hence 


sin A = a 2 A 2 + ai A + aol. 


(37) 
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Since À = 7t/2 is an eigenvalue of multiplicity 1 and X = n is an eigenvalue of 
multiplicity 2, it follows that 

fO r/2) = r{n/ 2), 
fin) = r(j, r), 

/(tt) = r'(jr); 

hence, 

sin7r/2 = Œ2(7t/2) 2 +û'i(7t/ 2) + ao, 

O 

sin 7r = «2 (tt) + ai(7r) + ao, 

COS7T = 2û' 2^T + oq, 


or simplifying 

4 = Oi2H + 2û'i7T + 4û'o, 

O 

0 = +Û'17T + Û'0, 

— 1 = 2a27r + ai. 


Thus, «2 = (1/tt 2 )(4 — 2n), a\ = (1/tt 2 )(— 8tt + 37r 2 ), ao = (l/7r 2 )(47r 2 — n 3 ). Sub- 
stituting these values into (37), we obtain 


sin A = 1 / 7ï 


2 




0 

16 - 10tt 


0 

0 



In closing, we point out that although exponentials of any square matrix can 
always be computed by the above methods, not ail functions of ail matrices can; 
/(A) must first be “well defined” whereby “well defined” (see Theorem 1 of Sec- 
tion 7.1) we mean that f(z) has a Taylor sériés which converges for \z\ < R and 
ail eigenvalues of A hâve the property that their absolute values are also less 
than R. 


Problems 7.5 

1. Establish the équations necessary to find e A if A is a 2 x 2 matrix having 1 and 
2 as its eigenvalues. 

2. Establish the équations necessary to find e A if A is a 2 x 2 matrix having 2 and 
2 as multiple eigenvalues. 

3. Establish the équations necessary to find e A if A is a 3 x 3 matrix having 2 as 
an eigenvalue of multiplicity three. 
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Functions of a Matrix 









11. 

12 . 

13 . 






Establish the équations necessary to find e A if A is a 3 x 3 matrix having 1, 
—2, and 3 as its eigenvalues. 

Redo Problem 4 if instead the eigenvalues are —2, —2, and 1. 

Establish the équations necessary to find sin(A) if A is a 3 x 3 matrix having 
1, 2, and 3 as its eigenvalues. 

Redo Problem 6 if instead the eigenvalues are —2, —2, and 1. 

Establish the équations necessary to find e A if A is a 4 x 4 matrix having 2 as 
an eigenvalue of multiplicity four. 

Establish the équations necessary to find e A if A is a 4 x 4 matrix having both 
2 and —2 as eigenvalues of multiplicity two. 

Redo Problem 9 if instead the function of interest is sin(A). 

Establish the équations necessary to find e A if A is a 4 x 4 matrix having 3, 3, 
3, and —1 as its eigenvalues. 

Redo Problem 11 if instead the function of interest is cos(A). 

Find e A for 


Find e A for 


Find e A for 


Find e A for 


Find cos A if 



3 

2 





1 2 
3 4 . 
0 2 



1 2 
-1 4 . 
0 2 



37T 

2tt 
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18. The function f(z) — log(l + z) has the Taylor sériés 


oo 


E 


(-1) k ~ 1 z k 
k 


which converges for |z| < 1. For the following matrices, A, détermine whether 
or not log(A + I) is well defined and, if so, find it. 


\\ il 


"-6 9" 


3 5“ 


"0 0 " 

.0 

(b) 

—2 3_ 

(c) 

-1 -3 

(d) 

0 0 


7.6 The Function e At 

A very important function in the matrix calculus is e At , where A is a square constant 
matrix (that is, ail of its entries are constants) and ris a variable. This function may 
be calculated by defining a new matrix B = At and then computing e B by the 
methods of the previous section. 


Example 1 Find e At if 



2 

3 


Solution Define 


B = Ar = 


t 

4 t 


2 t 
3 t ‘ 


The problem then reduces to finding e B . The eigenvalues of B are Ài = 5 r, À 2 = —t. 
Note that the eigenvalues now dépend on t. 


/(B) = e B r(B) = «iB + c^ol 

/(À) = e x r(X) = + afo- 

Since /(B) = r(B), 

e B = a\B + aol. (38) 


e 5t = a\(5t) + ao, 
e~' = ck,\ ( t) + ao. 


Also, f(Xi) = r(À/);hence 



7.6 The Function e 


A t 



Thus, ot\ — (1/6 t)(e 5t — e t ) and ao = (l/6)(e 5t + 5e *). Substituting these values 
into (38), we obtain 



1 2e 5t + 4e 1 2e 5t — 2e 1 
6 4e 5f - 4e~ r 4e 5r + 2éT' ’ 


Example 2 Find e At if 


Solution Define 



3 1 
0 3 
0 0 


0 

1 

3 


B = A t = 


3 1 
0 
0 


t 0 
3 1 t 

0 3 1 


The problem reduces to finding e B . The eigenvalues of B are 


Ai = À2 = A. 3 = 3 1 


thus, 


/(B) = e B r(B) = a 2B 2 + ai B + aol 
/(A) = e x r( A) = «2 A 2 + aiA + ao 



/(A) = /(A) = 2a 2 A + ai (40) 

f(X) = e x r"{\) = 2a 2 . (41) 

Since /(B) = r(B), 

e B = a 2 B 2 + ajB + aol- (42) 

Since A = 3ns an eigenvalue of multiplicity 3, 

/( 30 = r(3t), (43) 

f'(3t) = r'(3t), (44) 

f"(3t) = r" (3t). (45) 
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Thus, using (39)-(41), we obtain 


e 3t = (: 3t) 2 Œ2 + (3t)a\ + o?o, 


e 3 ' = 2(3t)d2 + ol i, 


e 3 ' — 2a 2 


or 


3/ = 9t 2 Œ2 + 3ta\ + c^o, 

(46) 

3/ = 6ta 2 + û'i, 

(47) 

3r = 2 c* 2 . 

(48) 


Solving (46)-(48) simultaneously, we obtain 

a 2 — \ e3î , oi\ = (1 — 3 t)e 3t , aQ = (1 — 3t + \ t 2 )e 3t . 

From (42), it folio ws that 



"9 1 2 

6 1 2 

r 2 “ 


~3î 

t 

0" 

e A ' = e B = te 3 ' 

0 

9 1 2 

6 1 2 

+ (1 - 3 t)e 3! 

0 

3 1 

t 

0 

0 

9 1 2 _ 


0 

0 

3 1 


+ (1-3 t+lt 2 )e 3t 


1 0 0 ' 
0 1 0 
0 0 1 


= e 


3 1 


'1 t î 2 /2 
0 1 t 

0 0 1 


Problems 7.6 


Find e At if A is given by: 







0 

0 

0 


1 0 

0 1 . 

0 0 



0 0 
1 2 . 
4 -1 
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7.7 Complex Eigenvalues 

When computing e At , it is often the case that the eigenvalues of B = At are 
complex. If this occurs the complex eigenvalues will appear in conjugate pairs, 
assuming the éléments of A to be real, and these can be combined to produce real 
functions. 

Let z represent a complex variable. Define e z by 



OO b 

Z K 


v 2 7 3 A v 5 

E 4 . Z Z Z Z 

k\ — + z + Al + 


k=0 


2! 3! 4! 5! 



(see Eq. (5)). Setting z = i6, 0 real, we obtain 



1 + iO + 


m 2 

2 ! ~ 



1 + i0- 



ie 3 e 4 w 5 
3T + 4! + 5! 


Combining real and imaginary terms, we obtain 




But the Maclaurin sériés expansions for sin 0 and cos 0 are 


sin 0 — — 


0 

Ï! 


e 3 e 5 

_ 3! + 5! “ 


cos 0 = 1 — 


e 2 o 4 e 6 

2! + 4! + 6! + 


hence, Eq. (50) may be rewritten as 

e ,e — cos 0 + i sin 0. 



Equation (51) is referred to as DeMoivre’s formula. If the same analysis is applied 
to z = —iO, it follows that 


e l0 = cos 0 — i sin 0. 
Adding (51) and (52), we obtain 



cos 0 = 


je 


+ e 


-iO 


2 


(53) 
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while subtracting (52) from (51), we obtain 


sin 6 = 


é e - e~ w 
2i 


( 54 ) 


Equations (53) and (54) are Euler’s relations and can be used to reduce 
complex exponentials to expressions involving real numbers. 


Example 1 Find e Xt if 


A = 


-1 5' 
-2 1 


Solution 


B = A t = 


‘ -t 5 t 
—2 t t 


Hence the eigenvalues of B are X\ = 3 ti and À 2 = —3 ti; thus 


/(B) = e 


B 


r(B) = ai B + aol 


/(À) = e x r(À) = ai À + ao 


Since /(B) = r(B), 


e B = aiB + agi, 


(55) 


and since /(A./) = r(À;), 


e 3/l = ai (3 ti) + ao, 


e 3,1 = ai (—3 ti) + ao. 


Thus, 


a 0 = 


ç3ti | g — 3 ti 
2 


1 / ^ 3 ti _ e -3ti 

and ai = — I 

3 1 \ 2 i 


If we now use (53) and (54), where in this case 0 = 3 1, it follows that 

ao = cos3r and ai = (l/3r)sin3r. 
Substituting these values into (55), we obtain 


= , B = 


r — j sin3r + cos3r 
— | sin3r 


| sin 3 1 

^ sin 3 1 + cos 3 1 
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In Example 1, the eigenvalues of B are pure imaginary permitting the 
application of (53) and (54) in a straightforward manner. In the general case, 
where the eigenvalues are complex numbers, we can still use Euler’s relations 
providing we note the following: 


g/H-ï'0 _|_ e P—iO 
2 


e Pe w 


+ e p e 




2 


e* (e w + e~ ie ) 
2 



cos 0, 



and 


çP+iO e P~ iO 

2 1 


2 / 


^ - e ~ ie ) 


2 i 


— e P sin 0. 


(57) 


Example 2 Find e At if 



Solution 


B = A t = 


2 1 
4 1 



hence, the eigenvalues of B are 





3 ,VÏ5\ 

— - t. 

2 2 J 


Thus, 


/(B) = e B r(B) = a\B + o?ol 
/(À) = r(À) = ai À + oro- 


Since /(B) = r(B), 



= aiB 4- aol, 


and since /(À/) = r(À z ), 

g[3/2+/(VÎ5/2)]< = ai[ 3 + i{ ^lS /2 )]t + a 0 , 
g[3/2— î(-v/Î5/2)]f = ai[ 3 _ Z(a /Ï5/2)]/ + oq. 
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Putting this System into matrix form, and solving for a\ and ao by inversion, we 
obtain 


ai = 


VÏ5 t 


>[(3/2)r+(VT5/2)ri] _ „[(3/2)?— (V / 15/2)r/] 


ao = 


-3 / e l(3/2)t+(y/Ï5/2)ti] _ g [(3/2)i-(VÏ5/2)ri] 

VÏ5 \ 2; 


,[(3/2)f+(VÏ5/2)ri] + e [(3/2)f-(VB/2)fl 


Using (56) and (57) where, fi = and 0 = (VÏ5/2 )t, we obtain 


2 3^/2 • 

^e l/ sin — - — 


Vl5r 


3 3^/2 • VÏ5 

— : r' sin 


VÏ5 


t + e 3//2 cos 


VÏ5 


Substituting these values into (58), we obtain 


- Jt/2 


1 . VÎ5 VÏ5 

— — = sin t + cos t 

VÎ5 2 2 

8 . VÏ5 

— = sin — — t 

VÎ5 2 


-2 . VÏ5 

—= sin 1 

VÎ5 2 

-1 . VÏ5 VT5 

— = sin — — t + cos — — -t 

Vl5 2 2 . 


Problems 7.7 

Find e At if A is given by: 




7.8 Properties of e l 
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7.8 Properties of e A 

Since the scalar function e x and the matrix function e A are defined similarly (see 
Eqs. (5) and (6)), it should not be surprising to find that they possess some similar 
properties. What might be surprising, however, is that not ail properties of e x are 
common to e A . For example, while it is always true that e x e y — e x+y — e y e x , the 
same cannot be said for matrices e A and e B unless A and B commute. 


Example 1 Find e A e B , e A+B , and e B e A if 


A ri ii a « r° ° 

A = 0 and B = 


Solution Using the methods developed in Section 7.5, we find 


a e e — 1 n 10 

— p n — p 

0 1’ 0 e 9 


A+B _ e e 
0 e 


Therefore, 




e e-ï\ [1 0 

0 1 0 e 


e e — e 


and 


e B , A = 


1 OlTe e — 1 


0 e 0 


e e — 1 
0 e 


hence 


e A+B ^ ^A+B ^ e B e A and e B e A ^ ^A^B_ 


Two properties that both e x and e A do hâve in common are given by the 
following: 

Property 1 e® = I, where 0 represents the zéro matrix. 

Proof. From (6) we hâve that 


00 / Ait 


k=0 


00 / A k 


k = 1 


Hence, 


00 fik 


= i+y- = i. 

b\ 


k = i 
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Property 2 ( e A ) 1 = e A . 

Proof 



Thus, e A is an inverse of e A . However, by définition, an inverse of e A is ( e A ) l ; 
hence, from the uniqueness of the inverse (Theorem 2 of Section 3.4), we hâve 
that e~ A — (e A ) -1 . □ 


Example 2 Verify Property 2 for 


Solution 


A = 


0 1 ' 
0 0 


—A = 


0 -1 

0 0 


e — 


1 1 
0 1 


and 




'1 

0 


-1 

1 


Thus, 


1 

"1 

r 

-i 

"i 


- p~ a 


0 

i 


0 

î 

— e 









Note that Property 2 implies that e A is always invertible even if A itself is not. 


Property 3 (e A ) T —e AT . 


Proof. The proof of this property is left as an exercise for the reader (see 
Problem 7). □ 
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Example 3 Verify Property 3 for 



Solution 


and 



1 

2 



1 2e 5 + 4e 1 
6 2e 5 — 2e~ l 



1 2e 5 + 4e 1 
6 4e 5 — 4^ _1 


Hence, (e A ) J 



Problems 7.8 


1. Verify Property 2 for 


2. Verify Property 2 for 


3. Verify Property 2 for 




A = 


'0 1 
0 0 
0 0 


What is the inverse of A? 

4. Verify Property 3 for 



J’ 

4e 5 — 4e~ l 
4e 5 + 2e~ 1 


2e 5 — 2e 1 
4e 5 4~ 2e~ l 


3 

1 ‘ 


1 

0 * 


0 

1 

0 


0 

0 . 
1 
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5. Verify Property 3 for the matrix given in Problem 2. 

6. Verify Property 3 for the matrix given in Problem 3. 

7. Prove Property 3. (Hint: Using the fact that the eigenvalues of A are identical 
to eigenvalues of A T , show that if e A = a n -iA n ~ l + • • • + a\ A + aol, and if 

e AT = &_!( A 7 )"- 1 + ■ ■ ■ + /6 iA t + fol. 


then a j — fij for j = 0, 1, . . . , n — 1.) 

8. Find e A e B , e B e A , and e A+B if 

0 1 
0 1 


A = 


1 1 
0 0 


and B = 


and show that £ A+B ^ e A eP, £ A+B ^ e B e A and e R e A ^ e A e B . 
9. Find two matrices A and B such that 


e 


A e B 


= , A+B 


10. By using the définition of e A , prove that if A and B commute, then e A e B = 
e A+B 

11. Show that if A = P _1 BP for some invertible matrix P, then e A = P _1 ^ B P. 


7.9 Dérivatives of a Matrix 


Définition 1 An n x n matrix A (t) = [aij(t)] is continuons at t — to if each of its 
éléments atj(t)(i, j — 1 , 2, . . . , n) is continuous at t — to. 

For example, the matrix given in (59) is continuous everywhere because each 
of its éléments is continuous everywhere while the matrix given in (60) is not 
continuous at t — 0 because the (1, 2) element, sin(l/r), is not continuous at t — 0. 

V t 2 - 1 

2 sin 2 1 



t 3 — 3t sin(l/0 
2 1 45 



We shall use the notation A (t) to emphasize that the matrix A may dépend on 
the variable t. 
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Définition 2 An n x n matrix A (t) — [aij(t) ] is différentiable at t — to if each of 
the éléments aij(t)(i, j = 1, 2, . . . , n) is différentiable at t = ro an d 



Generally we will use the notation A(Y) to represent dA(t)/dt. 


Example 1 Find A (t) if 



Solution 



Example 2 Find À(Y) if 



Solution 


r d(3t) 1 




Example 3 Find x(t) if 
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Solution 


Ht) = 


'H (0 

X2 (t) 

L x„(t) J 


The following properties of the dérivative can be verified: 
(pl) rf(A(Q+B(Q) = dA(t± + d»(0 


dt 


dt 


dt 


J[ûfA(0] dA(t) 

(P2) = a , where a is a constant. 




dt 


(P3) îWOAIOl = ( ^ AW + m ( JAwy „ hen Wt) is a scalar function 


dt 


\ Jr 


dt 


of t. 


<P4, rflA(, . lB(,)1 = (^) BU) + AU) (^ ). 


dt 


We warn the student to be very careful about the order of the matrices in (P4). 
Any commutation of the matrices on the right side will generally yield a wrong 
answer. For instance, it generally is not true that 


|[a W b W] = (^) B „) + (^'|aw. 


Example 4 Verify Property (P4) for 


A(0 = 


2 1 3 1 2 ' 

1 t 


and B (t) = 


1 2 1 
3 1 2 


Solution 


^[A(0B(0] 

dt 


d / 

"2r 3r 2 " 


"1 

2 1 

Jr \ 

1 t 


_3 1 

2 


J 

"2 1 + 9? 3 

1 

(N 

O 

t-H 


'2 + 27/ 2 20?" 


1 + 3? 2 

4 f _ 


6? 4 
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and 


dX(t) 

dt 


B(f)+A(f) 


~ dB(t)~ 


"2 6 1 


"1 

2 1 

+ 

"2 1 3 1 2 ' 


dt 


0 1 


3 1 

2 

JL t _ 



0 2 ' 
3 0 


'2 + lit 2 20 t 
6 t 4 

d[A(t)B(t)] 

dt 


We are now in a position to establish one of the more important properties of 
e Aî . It is this property that makes the exponential so useful in differential équations 
(as we shall see in Chapter 8) and hence so fundamental in analysis. 

Théo rem 1 If A is a constant matrix then 

de At 


dt 


Proof. From (6) we hâve that 


= Ae At = e Al A 


OO 


A t 


= E 

k=0 


(A t) 1 
k\ 


or 


A, _ . t 2 A 2 t 3 A 3 

e AI = I + t A + — — — + 


2 ! 


3! 


t n ~ 3 A. n ~ 3 t n A. n tn+l\n+l 

H - * • • + 

( n — 1)! n\ ( n + 1)! 


Therefore, 

de At 


dt 


A 2;A 2 3r 2 A 3 nt n ~ l A n ( n + l)t n A n+l 

oh 1 1 1 1 1 b 

1! 2! 3! n! (n + 1)! 


tA 2 t 2 A 3 t n ~ 1 A n t n A n + l 

= A+ — + — + •••+ 7- + — + 


1! 


2 ! 


(n — 1)! n\ 


tA t 2 A 2 r »-iA"-i t n A" 

1 + — + —T— + ••• + T” + — + 


1 ! 2 ! 


(n — 1)! n\ 


= ^ Ar A. 


If we had factored A on the left, instead of on the right, we would hâve obtained 
the other identity, 


de 


A t 


dt 


= Ae At . 


□ 
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Corollary 1 If A is a constant matrix, then 


de~ Al 

dt 




Proof. Define C = —A. Hence, e A/ = e Ct . Since C is a constant matrix, using 
Theorem 1, we hâve 


de^_ 

dt 




If we now substitute for C its value, — A, Corollary 1 is immédiate. □ 

Définition 3 An n x n matrix A (t) — [aij(t) ] is intégrable if each of its éléments 
aij(t)(i, 1,2,..., ai) is intégrable, and if this is the case, 


J A(0 dt = 



üij(t)dt . 


Example 5 Find f A(0 dt if 




Solution 

+ c\ 2t + C2 

^ 5 ^? 3 + c 3 e' + C4 


A (t) dt — 


f 3t dt f 2 dt 
f t 2 dt f e' dt 


Example 6 Find /q 1 A(f) dt if 



2 1 
è r 

sin nt 


1 


6t 2 

0 
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Solution 


A (t) dt = 





l 

2 dt 

— 1 dt 
1 

1 dt 



1 

2 

0 



The following property of the intégral can be verified: 

(P5) / 

where a and fi are constants. 


[aA(/) + /SB(r)] dt — a J A (t) dt + ji J B(f) dt. 


Problems 7.9 


1. Find Â(f) if 


(a) A (t) = 


cos t 
2 1 


t 2 - Il 
e (r-t) • 


(b) A(t) = 


2e' 3 4 5 

t 2 + 3t-l 
cos 3 (3f 2 ) 


t(t- 1) 17 

sin 2 1 t 
4 lnr 


2. Verify Properties (PI) - (P4) for 


ce = 7, P(t) = t 2 , 



and 




3. Prove that if dA(t)/dt — 0, then A (t) is a constant matrix. (That is, a matrix 
independent of t ). 

4 . Find J A(t) dt for the A (t) given in Problem l(a). 

5 . Verify Property (P5) for 


6 1 t 2 

2 1 1 ’ 


and 




a = 2, fi = 10, A (t) = 
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6. Using Property (P4), dérivé a formula for differentiating A 2 (t). Use this 
formula to find dA 2 (t)/dt, where 




and, show that dA 2 (t)/dt ^ 2A(t) dA(t)/dt. Therefore, the power rule of dif- 
férentiation does not hold for matrices unless a matrix commutes with its 
dérivative. 


7.10 Final Comments on Chapter 7 

We begin a proof of the Cayley-Hamilton theorem by noting that if B is an n x n 
matrix having éléments which are polynomials in À with constant coefficients, 
then B can be expressed as a matrix polynomial in À whose coefficients are n x n 
constant matrices. As an example, consider the following décomposition: 

"à 3 + 2A 2 + 37. + 4 2A 3 + 3A 2 + 4A + 5~ 

3A 3 + 4A 2 + 5 A 2k + 3 


"1 2" 


"2 3" 


"3 4" 


"4 5" 

3 0 

A 3 + 

4 0 

A 2 + 

5 2 

A + 

0 3 


In general, if the éléments of B are polynomials of degree k or less, then 

B = B^A^ B^-iÀ* + • • • + BiA + Bo, 

where B ; - (j = 0, 1, . . . , k) is an n x n constant matrix. 

Now let A be any arbitrary n x n matrix. Define 

C = (A - AI) (62) 


and let 


d(X) — X n + a n —\k n ! + ••• + a\k + ao (63) 

represent the characteristic polynomial of A. Thus, 

d(X) — det(A — AI) = det C. (64) 

Since C is an n x n matrix, it follows that the éléments of C a (see Définition 2 of 
Section 5.5) will be polynomials in À of either degree n — 1 or n — 2. (Eléments 



7.10 Final Comments 


on the diagonal of C a will be polynomials of degree n — 1 while ail other éléments 
will be polynomials of degree n — 2.) Thus, C a can be written as 

C a = C„_ir 1 F Q n - 2 ^ n 2 + • • • + CiÀ H- Co, (65) 

where C y - (y = 0, 1, . . . , n — 1) is an n x n constant matrix. 

From Theorem 2 of Section 5.5 and (64), we hâve that 

C a C = [detC]I = d(X)\. (66) 

From (62), we hâve that 

C a C = C a (A - AI) = C'A - XC a . (67) 

Equating (66) and (67), we obtain 

d(X) l = C a A-XC a . (68) 

Substituting (63) and (65) into (68), we find that 

\X n + a n —\\X n ^ + • • • + ^ilÀ + açjV 

= C„_iAà " _1 + C„_ 2 Aà "- 2 + • • • + Ci AA. + CoA 
-C„_iA." - C n - 2 X n ~ l CiÀ 2 - C 0 A. 

Both sides of this équation are matrix polynomials in À of degree n. Since two 
polynomials are equal if and only if their corresponding coefficients are equal we 
hâve 


I = ~Cn-l 

^hi — il — C/2—2 F C/2 — 1 A 


(69) 


a\\ — — Co F Ci A 
a 0 l = CqA. 


Multiplying the first équation in (69) by A”, the second équation by A” -1 , . . . , 
and the last équation by A 0 = I and adding, we obtain (note that the terms on the 
right-hand side cancel out) 

A" + 0 ,,-iA"- 1 + • • • + fll A + Aol = 0. (70) 

Equation (70) is the Cayley-Hamilton theorem. 
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Linear Differential Equations 


8.1 Fundamental Form 


We are now ready to solve linear differential équations. The method that we shall 
use involves introducing new variables x\ (t), X 2 (t ), . . . , x n (t ) , and then reducing a 
given System of differential équations to the System 


dx i (0 
dt 


= an (0*1 (0 + 012(0*2(0 H 1- ai n (t)x n (t) + /i(0 


dx2(t) 

dt 


= 021 (0*1 (0 + 022(0*2(0 H K a 2 n(t)x n (t) + / 2 (0 



dx n ( t ) 
dt 


= 0/il (0*1 (0 + 2 (0*2(0 H h 0/1/7 (0*n (0 + //i(0- 


If we define 



*1 (0 
*2(0 




011 (0 

012(0 

01/2 (0 


7i(o" 

A(0 = 

021 (0 

• 

• 

022(0 

• 

• 

••• 02/2 (0 

, and f(0 = 

/2(0 

9 

9 


• 

_a«l(0 

9 

0/(2 (0 

0/2/1 (0- 


9 

/«(0 



then (1) can be rewritten in the matrix form 


dx(0 


= A(0x(0 +f(0- 



dt 
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Example 1 Put the following System into matrix form: 

KO = t 2 y( t) + 3z(0 + sin t , 

KO = -e r y(t) + tz(t) — t 2 + 1. 

Note that we are using the standard notation y(0 and KO to represent 

dy(t) dz(t ) 

and . 

dt dt 

Solution Define jci (0 = KO and *2(0 — KO- This System is then équivalent to 
the matrix équation 


x\(t) 


' t 2 

3" 


X\ (t) 

+ 

sin t 1 

X2 (t)_ 


_-e r 

t 


_ x 2 (0. 

— i 
"rH 

+ 

(N 

1 


If we define 

shu 
-t 2 + l 


KO = 


x\ (0 
* 2(0 


A(0 = 


J 


and /(O = 



then (4) is in the required form, x(0 = A(0x(0 + f (0- ■ 

In practice, we are usually interested in solving an initial value problem; that 
is, we seek functions *i(0,*2(0, . ..,x n (t) that satisfy not only the differential 
équations given by (1) but also a set of initial conditions of the form 


*1 Où) = Ci, X2 (to) = C2, . . • , X n (to) — C n , 



where ci, C 2 , , c n , and to are known constants. Upon defining 


c 1 

C2 


C 


n 


it follows from the définition of x(0 (see Eqs. (2) and (5)) that 


x (to) = 


xi (to) 


ci 

X2 (to) 

• 

— 

C2 

• 

1 

* 

s 

iK 


• 

• 

_pn _ 


= c. 


Thus, the initial conditions can be put into the matrix form 


xOo) = C. 


( 6 ) 



8.1 Fundamental Form 


Définition 1 A System of differential équations is in fundamental form if it is 
given by the matrix équations 


x(r) = A (t)x(t) + f(0 

x(r 0 ) = c. (7) 


Example 2 Put the following System into fundamental form: 

x{t) — 2 x(t) — ty(t) 

ÿ(t) = * 2 *(0 + e' 
x(2) = 3, y (2) = 1 . 

Solution Define x\(t) — x{t) and X 2 (t) = y(t). This System is then équivalent to 
the matrix équations 


h ( t ) 


"2 

—t 


X \ (t) 

+ 

" 0 " 

X2 (0. 


J 2 

0 _ 


X2 (t)_ 

/_ 


x\(2) 


"3" 



1 


Consequently, if we define 





and to — 2, 


then (8) is in fundamental form. ■ 



Example 3 Put the following System into fundamental form: 

ï(t) = 2 l(t) + 3 m(t) — n(t) 
m (t) — l(t) — m(t) 
h(t) — m ( t ) — n ( t ) 


/ (15) =0, 


m (15) = -170, n (15) = 1007. 
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Solution Definexi(0 = /(O, *2(0 — m (t) , X 3 (t) — n{t). This System isthen équiv- 
alent to the matrix équations 


XI (t) 


"2 3 -1 


Xl(t) 

X2 (t) 

— 

1 -1 0 


X2 (Û 

* 3(0 


0 1 -1 


X3 (t) 


jci ( 15 ) 


0 

* 2 ( 15 ) 

— 

-170 

_ x 3 ( 15 )_ 


1007 


Thus, if we define 



XI (t) 


"2 

3 

-1 


" 0 " 

x(t) = 

X2 (t) 

, A (t) = 

1 

-1 

0 

, Ht) = 

0 


_X3 (t)_ 


0 

1 

-1 


0 


c = 


0 

-170 

1007 


and to — 15, 


then (9) is in fundamental form. ■ 



Définition 2 A System in fundamental form is homogeneous if f(0 = 0 (that 
is, if fi(0 = f 2 (0 = • • • = ï n (t) = 0) and nonhomogeneous if f(t) 7^ 0 (that is, if at 
least one component of f (t) differs from zéro). 

The System given in Examples 2 and 3 are nonhomogeneous and homogeneous 
respectively. 

Since we will be attempting to solve differential équations, it is important to 
know exactly what is meant by a solution. 

Définition 3 x(0 is a solution of (7) if 

(a) both x(t) and x(0 are continuous in some neighborhood 7 of the initial time 
t = to , 

(b) the substitution of x(t) into the differential équation 

x(0 = A(t)x(t) +f(0 

makes the équation an identity in t on the interval 7; that is, the équation is 
valid for each t in 7, and 

(c) x(fo) = c. 

It would also seem advantageous, before trying to find the solutions, to know 
whether or not a given System has any solutions at ail, and if it does, how 
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many. The following theorem from differential équations answers both of these 
questions. 

Theorem 1 Consider a System given by (7). If A(t) and f(t) are continuons in 
some interval containing t — îq, then this System possesses a unique continuons 
solution on that interval. 

Hence, to insure the applicability of this theorem, we assume for the remainder 
of the chapter that A (t) and f (t) are both continuous on some common interval 
containing t = t q. 
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du(t) 

dt 

dv(t) 

dt 

dw(t) 

dt 


— e t u(t) + tv(t) + w(t), 

— t 2 u(t) — 3 v(t) + (t + l)w(t), 
= v{t) + e' 2 w(t), 


u( 4) = 0, m( 4) = 1, z (4) = — 1. 



dyjt) 

dt 

dz(t) 

dt 


= 6 y(t) + zf, 

= x(t) — 3z(t), 
—2y(t), 


x(0) = 10, y( 0) = 10, zi 0) = 20. 


8. rit) — t 2 r(t) — 3 s(t) — (sin t)u(t) + sint, 

HO = HO - HO -ht 2 - 1, 

ü(t) — 2 rit) -h e f s(t) -h ( t 2 — 1 )u(t) -h cos t , 


r(l) = 4, j(l) = -2, w(l) = 5. 

9 . Détermine which of the following are solutions to the System 



snu 
cos t ’ 


il 


o r 


xi 


X, (O)' 


T 

*2_ 


-1 0 


_ x 2_ 

•> 

*2 (0). 


0 




cos t 
— sin t 


10 . Détermine which of the following are solutions to the System: 



il 


"1 

2“ 


xi 


*1 (0) 


T 

_i2_ 


4 

3 


_ x 2_ 

5 

_-*2(0)_ 


2 




V* " 

2e 5t * 


11. Détermine which of the following are solutions to the System: 


il 


0 

r 


xi 


x\ (1) 


T 

_i2_ 


-2 

3_ 


_ X 2_ 

•> 

_JC2 (1)_ 


0 



-e 2r + 2e r 
-2e lt + 2e t ’ 



- + 2e«-» " 

-2e 2(t - 1] + 2e (t ~ V) ’ 



V^- 1 ) 

o 



8.2 Réduction of an nth Order Equation 


ion of an nth Order Equation 


Before seeking solutions to linear differential équations, we will first develop 
techniques for reducing these équations to fundamental form. In this section, we 
consider the initial-value problems given by 


d n x(t) d n ^x(f) dx(t) 

a ti (0 " , n + a n-l(t) — 1 ■ H h Cl\ \f) ~ + ao(t)x(t) = f(t) 


( 10 ) 


*(*o) = ci, 


dx (to) 


= C2, . . • , 


d n 1 x (to) 


— Cn • 


Equation (10) is an nth order differential équation for x{t) where ao(t), 
a\{t ), . . . , a n (t) and fit) are assumed known and continuous on some interval 
containing îq . Furthermore, we assume that a n (t) 0 on this interval. 

A method of réduction, particularly useful for differential équations defined 
by System (10), is the following: 

Step i. Rewrite (10) so that the nth dérivative of x{t) appears by itself; 


d n x(t) 

dt n 


a n -i(t)d n 1 x(t) 
a n (t) dt n ~ l 


a\{t) dx(t) ao(t) f{t) 

x(t) H 

a n ( t ) dt a n ( t ) a n ( t ) 


(ii) 


Step 2. Define n new variables (the same number as the order of the differential 
équation), x\ (t), X 2 (t), , x n (t) by the équations 


= x(t), 
dx i 

= ~dT' 

dx 2 
= ~dt' 


( 12 ) 


dXn- 


Xfl — 1 — 


n—2 

dt 


dx n — i 


X n — 


Generally, we will write xj(t)(j = 1,2, ... ,n) simply as xj when the dependence 
on the variable t is obvious from context. It is immédiate from System (12) 
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that we also hâve the following relationships between *1, *2, . . . , and the 
unknown x(t): 


xi 



X 2 = 


x 3 




d 


n—2 


X 


Xn — 1 — 


X n = 


dt n ~ 2 
d "- 1 


X 


dt n 


-1 


Hence, by differentiating the last équation of (13), we hâve 


dx 


n 


dt 


d n x 

dt n 


Step 3. Re write dx n /dt in terms of the new variables x\,X2, 
Substituting (11) into (14), we hâve 


• ? X n 


(14) 


dx 


a n -\(t) d n 1 x 
a n (t) dt n 


n 


dt 


-1 


a\(t) dx ao(t) f{t) 

-x + 


a n (t) dt a n (t) a n (t) 


Substituting (13) into this équation, we obtain 


dx n 

dt 


an-lit ) 
a n (0 


X n 


aiit) a 0 (0 fit) 

— TT *2 —xi H — 

a n it) a n it) a n it) 


(15) 


Step 4. Form a System of n first-order differential équations for x\,X2, 
Using ( 12 ) and ( 15 ), we obtain the System 


» Xn • 


dx 1 
dt 
dx 2 
dt 


= X 2 , 


= * 3 , 


dx f i—2 

j = x n — 1 , 

dt 

dx n - 1 
dt =Xn ’ 

dx n aoit) a\it) 

= x\ X2 — 

dt a n (t) a n (t) 


a n -l(t) 
a n (. t ) 



f(t) 

a„ ( t ) 




8.2 Réduction of an nth Order Equation 


Note that in the last équation of (16) we hâve rearranged the order of (15) so that 
the xi term appears first, the X2 term appears second, etc. This was done in order 
to simplify the next step. 

Step 5. Put (16) into matrix form. 

Define 


and 


xi (0 
X2 (0 



X n -l(t) 

X n (0 



fit) = 


0 

0 

• 

0 


fit) 
a n (t) 


0 

0 

0 

1 

an-lit) 
a n (0 - 



Then (16) can be written as 


x(t) = A (t)\(t) + f(i). (18) 

Step 6. Rewrite the initial conditions in matrix form. 

From (17), (13), and (10), we hâve that 


x(*o) = 


xi (ro) 

T2 (^ 0 ) 


Xn (to) 


x(to) 
dx (^o) 
dt 

d n_1 x (to) 


ci 

C2 

Cr 


dt n ~ l 
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Thus, if we define 


ci 

C2 

Cn 


the initial conditions can be put into matrix form 


X (f 0 ) = c. 



Equations (18) and (19) together represent the fundamental form for (10). 

Since A (t) and f(t) are continuous (why?),Theorem 1 of the previous section 
guarantees that a unique solution exists to (18) and (19). Once this solution is 
obtained, x(r) will be known; hence, the components of x(t ), x\(t ), . . . , x n (t) will 
be known and, consequently, so will x(t), the variable originally sought (from (12), 
xi (0 = x(t)). 


Example 1 Put the following initial-value problem into fundamental form: 

2'x — 4x + 16tx — x + 2 t 2 x = sin t, 

x(0) = 1, x(0) = 2, x(0) = —1, x(0) = 0. 

Solution The differential équation may be rewritten as 


’x — 2 x — 8 tx + \x — t 2 x + s i n ^* 


Define 


x\ = x 

X 2 =X\= X , 


X3 = X2 = X, 


X4 = X3 = X 


hence, X 4 = x. Thus, 


-Il = X2 
X2 = X 3 


X 3 = X4 
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X4 — x — Zx — 8 tx + jx — t x + 2 sin t 

= 2x4 ~ 8^3 + ix2 — / 2 xi + i sin t, 


or 


Define 


XI = X2 

*2 = 

*3 = 

^ 1 

X4 = -rii + jX 2 


x 3 

X4 

8tX3 + 2x4 + \ sin t. 



1 0 0 

0 1 0 

0 0 1 ’ 

\ -8 1 2 


and = 0. 


Thus, the initial value problem may be rewritten in the fundamental form 

x(0 = A(0x(0 +f(0, 
x Oo) = c. ■ 


Example 2 Put the following initial value problem into fundamental form: 


x(2) = 1, 




Solution The differential équation may be rewritten 


d 5 x 



— te 1 x + 4. 
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Define 


hence, 


Thus, 


or 


Define 


x\ = x 
X2 — x\ — X 

X2—X2—X 
X 4 = X 3 = X 



d 4 x 



d 5 x 

dë' 


X\ = X2 
X2 = X 3 

*3 = x 4 
x 4 = x 5 


d 5 x ^ t d 4 x _ t 
X5 = — — = 2 e — — te r x + 4 
dt 5 dt 4 

= 2 e 1 x$ — te r x\ + 4 , 


il = 

*2 = 

*3 = 

X 4 — 

X5 = —te~'x i 


*2 

x 3 

X4 


x 5 

+ 2^x5 + 4 . 




— — 


0 

1 

0 

0 

0 " 

XI 


0 

0 

1 

0 

0 

*2 

, A (0 = 

0 

0 

0 

1 

0 

x 3 

0 

0 

0 

0 

1 

X 4 


—te~ 

r 0 

0 

0 

2e l 

_ x 5 _ 







"0" 


r 





0 


-1 





0 

, c = 

-1 

, and 

^0 

— 2. 

0 


2 





4 


3 






Thus, the initial value problem may be rewritten in the fundamental form 


x(0 = AO)x(r) + f(r), 
x (r 0 ) = c. ■ 



8.3 Réduction of a System 
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Problems 8.2 


Put the following initial-value problems into fundamental form: 


1. 


3 . 


5 . 


d 2 x dx 

—y - 2— - 3x = 0; 
dt z dî 

dx( 0) 

jc(0) = 4, — ^ = 5. 


dt 


— x — t 2 \ 


d z x 
dt 2 

'i dx (0) * 

x(0) = -3, — - — = 3. 

dt 

x — 3x + 2x = e ~ f , 
x(l) = i(l) = 2. 


2 . 


4 . 


6 . 4x 4- tx — x = 0, 

x(-l) = 2, x( — 1) = 1, x(-l) = -205. 


d 2 x . dx 

-tt + e t --tx = 0; 
dt 1 dt 

dx( 1 ) 

jc( 1) = 2, — ^ = 0. 


fl _ 2g“ r 

dt 2 


dt 

dx 

dt 


— 3e r x = 2; 


Jx(0) 

*(0) = 0, -^ = o. 

dt 


7 . 


8 . 


d 4 x d 2 x 


+ t- 


dt 4 ' dt 2 


= 1 + 


dx 
df ’ 


x(0) = 1, 


dx(0) 

dt 


— 2 , 


d 2 x(0) 

dt 2 


= TT, 


d 3 x(0) 

dt 2 


— e' 


d 6 x A d 4 x 0 

h 4 — t z — t 

dt 6 dt 4 

x(7T) = 2, x(7T) = 1, x(7T) = 0, x(7T) = 2, 


J 4 x( 7T) 

dt 4 


= 1 , 


J 5 x( 7T) 

dt 5 


= 0 . 


8.3 Réduction of a System 

Based on our work in the preceding section, we are now able to reduce Systems of 
higher order linear differential équations to fundamental form. The method, which 
is a straightforward extension of that used to reduce the nth order differential 
équation to fundamental form, is best demonstrated by examples. 


Example 1 Put the following System into fundamental form: 

x — 5x + y — 7y + e l , 

ÿ = x-2ÿ + 3y + smt, (20) 

jc (1) = 2, x (1) = 3, jc( 1) = — 1, y(l) = 0, y(l) = -2. 
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Step i . Rewrite the differential équations so that the highest dérivative of each 
unknown function appears by itself. For the above System, this has already 
been done. 

Step 2. Define new variables x\ (t), X 2 (t), x 3 (t), yi(t), and y 2 (t). (Since the highest 
dérivative of x(t) is of order 3, and the highest dérivative of y{t) is of order 2, 
we need 3 new variables for x(t) and 2 new variables for y(t). In general, for 
each unknown function we define a set a k new variables, where k is the order 
of the highest dérivative of the original function appearing in the System under 
considération). The new variables are defined in a manner analogous to that used 
in the previous section: 


x\ — x, 

X 2 = XI, 

X3 = X2, (21) 

yi = y , 
y2 = h- 

From (21), the new variables are related to the functions x(t) and y(t) by the 
following: 


x\ — x, 

X2 = x, 

X3 = x, (22) 

3^1 = 7 » 

T2 = y. 

It follows from (22), by differentiating X3 and y 2 , that 


x 3 = x, 




Step 3. Rewrite x 3 and y2 in terms of the new variables defined in (21). 
Substituting (20) into (23), we hâve 

x 3 = 5x + y - ly + e' , 
yi — x — 2y + 3y + sin t. 


Substituting (22) into these équations, we obtain 


x 3 = 5 x 3 + y2 - 7yi + e r , 
yi = X 2 - 2yi + 3yi + sin t. 


( 24 ) 
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STEP4. Set up a System of first-order differential équations for jq, X2, x 3 , yi, 
and y 3 . 

Using ( 21 ) and ( 24 ), we obtain the System 

xi = x 2 , 

X 2 = X 3 , 

xj = 5x3 - 7 yi + y 2 + e\ ( 25 ) 

h = n, 

h — x 2 + 3 vi - 2y 2 + sin t. 

Note, that for convenience we hâve rearranged terms in some of the équations to 
présent them in their natural order. 

Step 5. Write ( 25 ) in matrix form. 

Define 


x\ (t) 0 1 0 0 0 0 

X2 (t) 0 0 1 0 0 0 

x(0 = X 3 (t) , A (t) = 0 0 5 —7 1 , and f (t) = e l . (26) 

yi (t) 0 0 0 0 1 0 

y>2(t) 0 10 3 —2 sin^ 

Thus, Eq. (25) can be rewritten in the matrix form 

x(0 = A(r)x(0 + f (0- (27) 


Step 6 . Rewrite the initial conditions in matrix form. From Eqs. (26), (22), and 
( 20 ) we hâve 

xi(l) v(l) 2 

^ 2 ( 1 ) . 1 ( 1 ) 3 

x(l) = x 3 (l) = x(l) = -1 . 

yiO-) y( 1 ) 0 

_nd)J LkdJ L- 2 . 

Thus, if we define 

2 " 

3 

c = -1 
0 

-2 

and ro = 1 , then the initial conditions can be rewritten as 


x(l) = c. ■ 


(28) 
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Since A (t) and f(0 are continuons, (27) and (28) possess a unique solution. 
Once x(0 is known, we immediately hâve the components of x(t), namely 
x\ (t), X 2 (0 , X3 (t), y\(t) and y 2 (t ) . Thus, we hâve the functions x(t) and y(t) (from 
(21), xi (t) = x(t) and y\{t) = y(t)). 

Ail similar Systems containing higher order dérivatives may be put into 
fundamental form in exactly the same manner as that used here. 


Example 2 Put the following System into fundamental form: 

x = 2x + ty — 3z + t 2 z 4- t, 
y = z + (sin t) y -\- x - t, 

Z = x-ÿ + t 2 + l; 

x(n) = 15, x{n) = 59, x(jt) = — 117, y(n)=2, y(n) = —19, 

ÿ ( jt ) =3, z (7r) = 36, z (7r) = —212. 

Solution Define 

x\ = x 
X 2 — x\ = X 

x^ — X 2 — ’x; hence, x 3 = x. 

yi = y 

y2 = yi = y 

73 = y2 = ÿ; hence, y 3 = y. 
zi — z 

Z 2 = zi = z; hence, ^2 = 2. 

Thus, 

il = X2 

X2 = X 3 

_ O 

X 3 = x = 2 x, + ty — 3z + rz + t 
= 2 x 2 + ty 2 - 3z\ + t 2 Z 2 + f; 
ii = yi 


y2 = b 



8.3 Réduction of a System 
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B = y = Z + (sin t)y + x - t 

= Z2 + (sin t) y\ + x\ - t\ 

Zl = Z2 

Z2 = Z = X - ÿ + r + 1 

= b - B + 1 2 + 1; 
or 


XI = 

X2 



*2 = 

X3 



x 3 = 

2x2 

+ ty 2 

-3zi + t 2 z2 + t 

yi = 


y2 


h = 


n 


ÿ 3 = x\ 

+ (sin t ) y\ 


Z2 ~ t 

zi = 



Z2 

Z2 = 

x 3 

-y 3 

+ t 2 + 1 


Define 

roio o oooo" 

JC 2 00100000 

x 3 0 2 0 0 t 0 —3 t 2 

_ yi X(t ,_ ooo o îooo 

y 2 ’ 0 0 0 0 0 1 0 0’ 

y 3 1 0 0 sin t 0 0 0 1 

zi 00000001 

_z 2 J [o 0 1 0 0 -1 0 0_ 



Thus, the System can now be rewritten in the fundamental form 

x(t) = A (0x0) +f(0, 
x Oo) = c. ■ 
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Problems 8.3 

Put the following initial-value problems into fundamental form: 


1. 


d z x 
dt 2 
dy 
dt 


„dx 

2— + 3x + 4y , 
dt 


— 5x — 6y, 


*(0) = 7, 


dx( 0) 


= 8, 


dt 

y( 0) = 9. 


2 . 


3 . 


d z x 
dt 2 

d 2 y 
dt 2 

x(0) 

dx 


dx dy 

— — h — , 
dt dt 

dy dx 


dt 

2 , 


dt 

d 2 y 


= r 




d^ ’ 
dx(0) 
dt 

— 4x , 


dy(0) 

3, y(0) = 4, = 4. 

dt 


— ty + ^ x, 


dt 2 

x(2) — — 1, y(2) = 0, 


4 . 


dx 

d? 

d 2 y 

df 2 


2 ? - 4 , + ,, 

dt 


— ty 4~ 3x — 1 , 


<fo(2) 

dt 


= 0. 


x(3) = 0, y(3) = 0, 


dy( 3) 
dt 


= 0 . 


5. x = 2x + ÿ-*, 

•••• •• # 

y = tx — ty + y — e ; 

x(-l) = 2, i(-l) = 0, y(— 1) = 0, ÿ(— 1) = 3, ÿ(-l) = 9, 

y(-l) = 4. 



8.4 Solutions of Systems with Constant Coefficients 


6. x = x — y + y, 
ÿ = x — x + 2ÿ; 

*(0) = 21, *(0) = 4, *(0) = —5, y(0) = 5, y(0) = 7. 

7. x = y — 2, 
ÿ = z-2, 
z — x -\- y\ 

x(jt) = 1, y(7T) = 2, y(7T) = 17, z( n) = 0. 

8. X = y + z + 2, 
ÿ = x + y - 1, 
z = x - z H- 1; 

*(20) = 4, *(20) = -4, 3^(20) = 5, y(20) = -5, z(20) = 9, 

z(20) = -9. 

8.4 Solutions of Systems with Constant Coefficients 

In general, when one reduces a System of differential équations to fundamental 
form, the matrix A (t) will dépend explicitly on the variable t. For some Systems 
however, A (t) does not vary with t (that is, every element of A (t) is a constant). 
If this is the case, the System is said to hâve constant coefficients. For instance, 
in Section 8.1, Example 3 illustrâtes a System having constant coefficients, while 
Example 2 illustrâtes a System that does not hâve constant coefficients. 

In this section, we only consider Systems having constant coefficients; hence, 
we shall designate the matrix A (t) as A in order to emphasize its independence 
of t. We seek the solution to the initial-value problem in the fundamental form 

x(t) = A x(t) + f (0, (29) 

x(to) = c. 

The differential équation in (29) can be written as 

x(t) — A x(t) = f (t). (30) 

If we premultiply each side of (30) by e~ At , we obtain 

e~ At [x(0 — Ax(0] = e~ At f (t). (31) 

Using matrix différentiation and Corollary 1 of Section 7.9, we find that 

= e~ A ' (-A)x(f) + e~ At x(t) 

= e~ At [x(0 — Ax(0] • 


(32) 
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Substituting (32) into (31), we obtain 


[e“ Af x(r)] = e~ A, t (r). 


(33) 


Integrating (33) between the limits t — ?o and t — t, we hâve 


J je A/ x(oJ dt — J e A/ f(0 dt 


(34) 


— le As f (5) ds. 

JtQ 


Note that we hâve replaced the dummy variable t by the dummy variable 5 in 
the right-hand intégral of (34), which in no way alters the definite intégral (see 
Problem 1). 

Upon evaluating the left-hand intégral, it follows from (34) that 


t r t 

e~ At x(t) = / e~ As f(s)ds 
tQ J r 0 


or that 


e At x(t) = e A/ °x(^o) + f e As f {s)ds. 

J ta 


(35) 


But x(fo) = c, hence 


e~ A, x(t) = 


= e~ A,0 c+ [ e~ As f (s)ds, 
J ta 


(36) 


Premultiplying both sides of (36) by ( e Ar ) \ we obtain 


x(r) = (e~ At )~ 1 e~ At °c + (e~ At )~ l / e As î (s)ds. 


(37) 


Using Property 2 of Section 7.8, we hâve 


AA-1 


)- l =e A \ 


Whereupon we can re write (37) as 


x(t) = e A, e~ A, °c + e At f e~ As f (s) ds. 

J ta 


( 38 ) 



8.4 Solutions of Systems with Constant Coefficients 


Since A t and — A^o commute (why?), we hâve from Problem 10 of Section 7.8, 


e At e ~ At ° = e A(t ~ to) 



Finally using (39), we can rewrite (38) as 


x(0 = 


e A(t-t 0 ) c 


+ e At I e As f (s)ds. 



Equation (40) is the unique solution to the initial-value problem given by (29). 

A simple method for calculating the quantities e A ^ r ~ t0 \ and e~ As is to first 
compute e At (see Section 7.6) and then replace the variable t wherever it appears 
by the variables t — îq and (— s), respectively. 


Example 1 Find e A(t ?o) and e As for 


1 

0 -1 


Solution Using the method of Section 7.6, we calculate e At as 


Hence, 



e 1 te 1 

0 * 


e Mt-t 0 ) _ 


0 


(t - to) e- (t ~ ,o) 


and 



Note that when t is replaced by ( t — fy) in e~\ the resuit is e -(r -/o) = e~ î+tQ and 
not That is, we replaced the quantity t by the quantity (t — ?o); we did 

not simply add — to the variable t wherever it appeared. Also note that the 
same resuit could hâve been obtained for e~ As by first computing e As and then 
inverting by the method of cofactors (recall that e~ As is the inverse of e As ) or 
by computing e~ As directly (define B = — As and calculate <? B ). However, if 
e At is already known, the above method is by far the most expédient one for 
obtaining e ~ As . 
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We can dérivé an alternate représentation for the solution vector x(t) if we 
note that e At dépends only on t and the intégration is with respect to Hence, e At 
can be brought inside the intégral, and (40) can be rewritten as 


x(0 = 


e A (/— ? 0 ) c 


+ / e A 'e~ As f (s)ds. 


Since A t and — As commute, we hâve that 


e At e~ As = e A(t ~ s) 


Thus, the solution to (29) can be written as 


x(t) — e A{t /o) c + 



e A(t ~ s) î(s)ds. 



Again the quantity ^ A(r - Ç ) C an be obtained by replacing the variable t in e At by 
the variable (t — s). 

In general, the solution x(t) may be obtained quicker by using (41) than by using 
(40), since there is one less multiplication involved. (Note that in (40) one must 
premultiply the intégral by e Aî while in (41) this step is eliminated.) However, 
since the intégration in (41) is more difficult than that in (40), the reader who 
is not confident of his integrating abilities will probably be more comfortable 
using (40). 

If one has a homogeneous initial-value problem with constant coefficients, that 
is, a System defined by 


x(t) — Ax(r), 
X (to) = c, 



a great simplification of (40) is effected. In this case, f (t) = 0. The intégral in 
(40), therefore, becomes the zéro vector, and the solution to the System given 
by (42) is 


x(t) = e A{t ~ to) c. (43) 

Occasionally, we are interested in just solving a differential équation and not 
an entire initial-value problem. In this case, the general solution can be shown to 
be (see Problem 2) 


x(t) = e A 'k + e 


A t 


/ 


—A t 


f (t) dt , 



where k is an arbitrary n-dimensional constant vector. The general solution to the 
homogeneous differential équation by itself is given by 

x(?) = e At k. 


(45) 



8.4 Solutions of Systems with Constant Coefficients 
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Example 2 Use matrix methods to solve 

à(t) = u(t) + 2 v(t) + 1 
v(t) — 4 u(t) + 3v(t) — 1 
u( 0) = 1, v(0) = 2. 

Solution This System can be put into fundamental form if we define to — 0, 

= [“!?!, a = I"! fco = T 11- and c=Rl- ( 46 ) 


Since A is independent of t , this is a System with constant coefficients, and the 
solution is given by (40). For the A in (46), e At is found to be 




2e 5 ' + 4e~‘ 2e 5t - 2e~ r 
4e 5 ' - 4e - ' 4e 5 ' + 2e“' 


Hence, 


= 


1 


2e~ 5s + 4e s 2e~ 5s - 2e s ' 
4e~ 5s - 4e s 4e~ is + 2e s 


and 


Thus, 


g A(r-r 0 ) = e At, since = q 


e A(t-t 0 ) _ 1 

6 


'2e 5 ' + 4e“' 2e 5t - 2e~ r 
4e 5 ' - 4e~< 4e 5 ' + 2e“' 



ru 


— i 
(N 


1 

6 


1 [2e 5t + 4e“'] + 2 [2e 5 ' - 2e“'] 
|_1 [4e 5 ' - 4e“'] + 2 [4e 5 ' + 2e“'] 


,5 1 


2e 


5 1 


( 47 ) 


and 


—As 


f (S) = 


2e~ 5s + 4e s 
4e~ Ss - 4e s 


2e~ 5s - 2e s 
4e _5s + 2e s 



i 


-1 


1 

6 


1 [2e~ Ss + 4e s ] - 1 [2e“ 5s - 2e s ] 
|_1 [4e~ 5s - 4e s ] - 1 [4e- 5 - 5 + 2e ç ] 


—e 
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Hence, 



e As f (s)ds = 




1 

1 


1 

r— < 

1 

1 

1 

1 

to 

©*•* 

1 


T— < 

T 

1 

1 


and 




e As f(s)ds 


1 2e 5t + 4e~ f 
6 4e 5t - 4e~ r 


2e 5 ’ - 2e~ r 


1 

r-H 

1 

■4— 

1 

4e 5 ' + 2e~' 


1 

1 

r-H 

1 


! [2e 5t + 4e~ r ] [ e * - 1] + [2e 5t - 2e~ f ] [1 - é\ 
6 [ 4e 5 > _ 4^-1] [gf _ 1] + [4e 5 ' + 2e~<] [1 - e'] 


(l-e ')' 

L(-i+‘-')J' 


Substituting (47) and (48) into (40), we hâve 


1 1 

S V 

1 1 

= x(0 = 

1 1 

■*— I/o 

1 1 

+ 

1 — e~ r 

r 1 + e ~‘. 

— 

e 5t 4- 1 — e 1 
2e 5t — 1 + e~ l 



or 


u(t) = e 5t — e 1 + 1, 
vit) = 2e 5 ' + e~' - 1. ■ 


Example 3 Use matrix methods to solve 


ÿ — 3ÿ 4- 2y = e - , 
y (1) = 1, j(l) = 0. 


Solution This System can be put into fundamental form if we define to = 1; 


x(0 = 


X\ (t) 
_X2 (f)_ 

, A = 

0 1" 

-2 3 

. m = 

i i 

0 | 

1 i 

, and c = 

T 

0 


The solution to this System is given by (40). For this A, 



— e 2t 4~ 2e l e 2î — e 1 
-2e lt +2e t 2e 2t - é * 



8.4 Solutions of Systems with Constant Coefficients 


281 


Thus, 


e Mt-to) c — 


' _ e 2(r-l) + 2 e<t- 1) 
— 2e 2(r_1> + 2e^ 


e 2(r-l) _ g 
2e 2(r_1) - e 


(t-l)' 

(t-l) 



1 

h- 1 

1 


1 

O 

1 


■ — g2(î— l) + 2e(t-\ y 
_ 2e 2(r-l) + 2e (»-l) 



Now 



and 


e- As fCv) = 


— e -2s + 2e _s 
— 2e _2s + 2e~ s 




e -5s _ e -4.r 
2 É .- 5i - e - 4 ' ' 


Hence, 



e As f (s)ds — 



and 






A t 


f 

Jto 


e As f(^)ûfi 


(-e 2 ' + 2e ') (e 2 ' - e 1 ) 

l(-2e 2, + 2e') (2e 2 ' - e 1 ) 


J_ e -3 1 , lJ2t-5) _ lj-4 
20 e + 5 e 4 e 

3_ e ~3t , 2 (2f-5) _ 1 t-4 
20 e + 5 e 4 e 


le~ 5t + \e- 4 ‘ + 


5 

2 e ~5t 
5 e 


+ l 4 e- 4 ' + f e - 5 


l e ~4 
4 e 

V 4 

4 e 


( 50 ) 
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Substituting (49) and (50) into (40), we hâve that 



X] (t) 


x 2 (t)_ 



- e 2{, ~ l) + 2é~ l 
-2e 2i, - l) + 2ë~ l 


± e -3t , l g (2f— 5) _ lgt—4 
20 e ^ 5 e 4 e 

_ 3_ -3r , 2 (2f-5) _ l„/-4 
20 e ^ 5 e 4 e 


— e 2 ( f_1 > + 2e f_1 + + ie( 2r “ 5 ) - 

— 2e 2(f_1> + 2e r_1 - ^e~ 3t + fet 2 '” 5 ) - \e r ~ 4 


Thus, it follows that the solution to the initial-value problem is given by 


y(t) = x\ (0 


= _^-D + 2^-1 + ^ + 1^-5) 


1^-4 


The most tedious step in Example 3 was multiplying the matrix e A/ by the 
vector fl e~ As f(s)ds. We could hâve eliminated this multiplication had we used 
(41) for the solution rather than (40). Of course, in using (41), we would hâve had 
to handle an intégral rather more complicated than the one we encountered. 

If A and f (t) are relatively simple (for instance, if f (t) is a constant vector), then 
the intégral obtained in (41) may not be too tedious to evaluate, and its use can 
be a real savings in time and effort over the use of (40). We illustrate this point in 
the next example. 


Example 4 Use matrix methods to solve 


x(t) 4- x(t) — 2, 
x (n) — 0, x (n) = — 1. 

Solution This initial-valued problem can be put into fundamental form if we 
define to — n , 



x \ (t) 
X2 (t)_ 

, A = 

1 1 

1 

o 

O 

1 1 

, Ht) = 

"0" 

2 

, and c = 

0" 

-1 



Here, A is again independent of the variable t , hence, the solution is given by 
either (40) or (41). This time we elect to use (41). For the A given in (51), e Al is 
found to be 



cos t sin t 
— sin t cos t 
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Thus, 


and 


e A(t-to) c _ 


cos ( t — 7 r) 
— sin (t — tt) 


sin (f — 7 i ) 


0 " 

cos (f — tt) 


-1 


— sin (t — tt) 

— COS (t — 7t) ’ 





cos ( t — s) 
— sin (t — s) 


sin (r — s) 


"()' 

cos (r — s) 


2 


2 sin (t — s) 
2 cos (t — s) 


Hence, 



e Mt - s) t(s)ds = 



2 sin ( t — s) ds 
2 cos ( t — s)ds 


2 — 2 cos ( t — tt) 
2 sin (t — 7r) 


(53) 


Substituting (52) and (53) into (41) and using the trigonométrie identities 
sin (t — tt ) — — sin t and cos (t — tt ) — lir,uo 


x\(t) 
*2 (0 



— sin ( t — n) 

— cos (t — tt) 


+ 


2 — 2 cos ( t — tt) 
2 sin ( t — tt) 


_ sin t + 2 cos t + 2 
cos ^ — 2 sin t 

Thus, since x(t) — x\ (t), it follows that the solution to the initial-value problem is 
given by 


x(t) = sin t + 2 cos t + 2. ■ 


Example 5 Solve by matrix methods 


m(0 = m( 0 + 2u(0, 
v(t) = 4u(t) + 3i>(0. 
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Solution This System can be put into fundamental form if we define 




1 2 
4 3 ’ 


and f (t) — 


0 

0 


This is a homogeneous System with constant coefficients and no initial conditions 
specified; hence, the general solution is given by (45). 

As in Example 2, for this A, we hâve 


Thus, 


A/ 1 
e =6 


'2e 5 ' + Ae~’ 2e 5 ' - 2e~ r 
4e 5 ' - Ae~' Ae 5 ' + 2e~' 


At 


k =i 



ki 


_k 2 _ 


1 

6 

1 

6 


2e 5t + 4e~ r 2e 5t - 2e~‘ 
4e 5 f - 4e~ { 4e 5r + 2é>~ r 


ki [2e 5t + 4e~ f ] + k 2 [ 2e 5t - 2e~ ! ] 
\_k\ [4e 5t — 4e ~ l ] + k 2 [4e 5t + 2e~ l \ 

e 5t (2k\ + 2k 2 ) + e~ ! (4ki ~ 2k 2 ) 
e 5t (4ki 4- 4k 2 ) + e~ t (—4ki + 2 ^ 2 ) 


Substituting (54) into (45), we hâve that 


m(0 

v(t) 



1 e 5/ (2£i + 2 ^ 2 ) + e ' (4ki ~ 2k 2 ) 

6 £ 5/ (4&i + 4 ^ 2 ) + e~ t (— 4ki + 2 ^ 2 ) 


or 


„(» = 1 2 ti t 2tz N i + ( 4ti - 2t; 1 


„(„ = 2 1 2 h ± A k A \ + ( e -, 




We can simplify the expressions for u (t) and v(t) if we introduce two new arbitrary 
constants k 3 and k$ defined by 



2ki 4- 2k2 



4ki - 2k 2 
6 


Substituting these values into (55), we obtain 


u(t) = k 3 e 5t + k\e 1 
v(t) = 2k 3 e 5t — k^e~' . ■ 



( 57 ) 
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Problems 8.4 

1. Show by direct intégration that 



In general, show that if f(t) is intégrable on [ a , /?], then 



/O) ds. 


(Hint: Assume f f(t) dî — F(t ) + c. Hence, f f(s ) ds — F (s) + c. Use the funda- 
mental theorem of intégral calculus to obtain the resuit.) 

2. Dérivé Eq. (44). (Hint: Follow steps (30)-(33). For step (34) use indefinite 
intégration and note that 



dt — e At x(t) + k, 



where k is an arbitrary constant vector of intégration.) 
Find (a) e~ At (b) e A d~ 2 ) (c) e A d~ s ) (d) if 




t t 2 /2 
1 t 
0 1 


4 . For e At as given in Problem 3, invert by the method of cofactors to obtain e At 
and hence verify part (a) of that problem. 

5. Find (a) e ~ At , (b) e ~ As , (c) e A h -3 ) jf 




2e 51 + 4e~ f 2e 5t - 2e~ f 
4e 5 ' - 4e~' 4e 5 ' + 2e~' 


(b) e- A -\ (c) if 




— sin 3t + 3 cos 3 1 
— 2sin3r 


5 sin 3 1 

sin 3t + 3 cos 3r 


6. Find (a) e A/ , 
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Solve the Systems given in Problems 7 through 14 by matrix methods. Note that 
Problems 7 through 10 hâve the same coefficient matrix. 


7 . KO — —2 KO + 3 y(t), 

KO = -KO + 2 y(0; 

x(2) = 2, y(2) = 4. 


8 . KO — —2 KO + 3y(0 + 1, 

KO = -x(t)+2y(t) + 1; 

Kl) = 1, y(l) = 1. 


9 . KO — —2x(t) + 3y(0, 
KO = -x(0 + 2y(0- 

11. KO = —4x(t) + sin 
KO) = 1, KO) = o. 

13 . x — x — 2x — e ~ l ; 

KO) = 1, KO) = 0. 


10 . x(t) — —2x(t) + 3 KO + 1? 

KO = —x(t) + 2y(0 + 1. 

12 . KO — ^ 

x(l) = 1, x(l) = 2, Kl) — 3 

14 . x = 2i + 5y + 3, 
y = -i - 2y; 

x(0) = 0, i:(0) = 0, y(0) = 1. 


8.5 Solutions of Systems — General Case 

Having completely solved Systems of linear differential équations with con- 
stant coefficients, we now turn our attention to the solutions of Systems of 
the form 


x(0 = A(0x(0 + f(0, 

X(fo) = c. (58) 

Note that A(0 may now dépend on t , hence the analysis of Section 8.4 does not 
apply. However, since we still require both A(0 and f (0 to be continuous in some 
interval about t — to, Theorem 1 of Section 8.1 still guarantees that (58) has a 
unique solution. Our aim in this section is to obtain a représentation for this 
solution. 

Définition 1 A transition (or fundamental) matrix of the homogeneous équa- 
tion x(0 = A(0x(0 is an n x n matrix 4>(f, to) having the properties that 

(a) K(Mo) = AO)$(Uo), (59) 

dt 

(b) fc(f 0 Ko)=I- (6°) 

Here to is the initial time given in (58). In the Final Comments to this chapter, we 
show that $(f, to) exists and is unique. 
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Example 1 Find <I>(7, to) if A(f) is a constant matrix. 

Solution Consider the matrix e A{t ~ t0 \ From Property 1 of Section 7.8, we hâve 
that e A{t °~ to) = e° = I, while from Theorem 1 of Section 7.9, we hâve that 

- c A«-‘o> = 1 ( e ^ e - A ‘o) 
dt dt V / 

= Ae At e~ At ° = 

Thus, e A ^~ tQ>) satisfies (59) and (60). Since <I>(£, to) is unique, it follows for the case 
where A is a constant matrix that 

<b(t,t 0 ) = e Mt - to) . ■ (61) 


Caution. Although $(r, to) = e A ^~ t0 ^ if A is a constant matrix, this equality is 
not valid if A actually dépends on t. In fact, it is usually impossible to explic- 
itly find Q>(t, to) in the general time varying case. Usually, the best we can say 
about the transition matrix is that it exists, it is unique, and, of course, it satisfies 
(59) and (60). 

One immédiate use of $(f, to) is that it enables us to theoretically solve the 
general homogeneous initial-value problem 

x(t) = A (t)x(t) 

x (f 0 ) = c. (62) 


Theorem 1 The unique solution to (62) is 

x(t) = &(t, to) C. (63) 

Proof. If A (t) is a constant matrix, (63) reduces to (43) (see (61)), hence 
Theorem 1 is valid. In general, however, we hâve that 

dx(t) d r , d r 

—TT = -T. [*(t, to)c] = - [*(t, to)]c, 

dt dt dt 

— A(t)<b(t , to)c {from (59), 

= A (t)x(t) {from (63), 


and 


x(t o) = &(to, to)c, 

= Ic 


{from (60) , 
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Example 2 Findx(r) andy(^) if 


x — ty 



—tx 


*(1) = 0, y(X) = 1, 


Solution Putting this System into fundamental form, we obtain 



A(0 = 


0 t 
-t 0 


f (0 = 0, c = 


' 0 ' 

1 


and 

x(0 = A (0x(0, 
x(to) = c. 

The transition matrix for this System can be shown to be (see Problem 1) 


$0, t 0 ) = 



sin 



cos 



Thus, from (63), we hâve 



sin 




Consequently, the solution is 



y(t) = cos 




2 




x(t) = sin 
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The transition matrix also enables us to give a représentation for the solution 
of the general time-varying initial-value problem 

x(t) = A (t)x(t) + f(0, 

X(fo) = C. (58) 

Théo rem 2 The unique solution to (58) is 

x(t) = <I>(r, to)c + f &(t, s)f(s)ds. (64) 

Jto 

Proof. If A is a constant matrix, &(t, to) = hence s) = e A ^~ s>) and 

(64) reduces to (41). We defer the proof of the general case, where A (t) dépends 
on t , until later in this section. 

Equation (64) is the solution to the general initial-value problem given by (58). 
It should be noted, however, that since 4>0, to) is not explicitly known, x(t) will not 
be explicitly known either, hence, (64) is not as useful a formula as it might first 
appear. Unfortunately, (64) is the best solution that we can obtain for the general 
time varying problem. The student should not despair, though. It is often the case 
that by knowing enough properties of &(t, to ), we can extract a fair amount of 
information about the solution from (64). In fact, we can sometimes even obtain 
the exact solution! □ 

Before considering some important properties of the transition matrix, we State 
one lemma that we ask the student to prove (see Problem 3). 

Lemma 1 7/B(0 is an n x n matrix having the property that B (Oc = 0 for every 

n-dimensional constant vector c, then B(0 is the zéro matrix. 

For the remainder of this section we assume that 4>(r, fo) is the transition matrix 
for x(t) = A (t)x(t). 

Property 1 (The transition property) 

$(f, T)$(T, fo) = $(f, fo)- (65) 

Proof. If A (t) is a constant matrix, <t>(r. to) = e A(f ~‘ 0> hence, 

<ï>(r, r)$(r, t 0 ) = e A(f “ T >e A(T - f o) 

_ e A(t-T+r-to ) 


= e Mt ~ to) = *(t, to). 


□ 
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Thus, Property 1 is immédiate. For the more general case, that in which A (t) 
dépends on t , the argument runs as follows: Consider the initial-value problem 


x(r) = A (t)x(t) 

x(r 0 ) = c. (66) 

The unique solution of (66) is 

x(f) = $(f, t 0 )c. (67) 


Hence, 


x(q) = $(fi, to)c (68) 

and 

x(r) = <fe(r, f 0 )c, (69) 


where t\ is any arbitrary time greater than r. If we designate the vector x(ti) by d 
and the vector x(r) by b, then we can give the solution graphically by Figure 8.1. 
Consider an associated System governed by 


x(0 = A(0x(0, 

x(r) = b. (70) 

We seek a solution to the above differential équation that has an initial value b at 
the initial time t — r. If we designate the solution by y(f), it follows fromTheorem 1 
that 

y (0 = r)b, (71) 


hence 


y (fi) = $(fi, t)b. 




Figure 8.1 
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But now we note that both x(t) and y(t) are governed by the same équation of 
motion, namely x(t) = A (t)x(t), and both x(t) and y (t) go through the same point 
(r, b). Thus, x(t) and y(r) must be the same solution. That is, the solution curve for 
y (y) looks exactly like that of x(r), shown in Figure 8.1 except that it starts at t = r, 
while that of x(t) starts at t = to. Hence, 


x(0 = y(0, t > r, 


and, in particular, 


x(*i) = y(T). 



Thus, substituting (68) and (72) into (73) we obtain 


*(fi,fo) c = $(*i, r)b. 



However,x(r) = b, thus (74) may be rewritten as 


to)c = &(ti, r)x(r). 



Substituting (69) into (75), we hâve 


$(T, *o)c = $(T, t)$(t, t 0 )c 


or 


[<ï>(q, to) - r)$(r, to)]c = 0. (76) 

Since c may represent any n-dimensional initial State, it follows from Lemma 1 
that 


<ï>(q, to) - r)$(r, fo) = 0 


or 


<ï>(q, to) = $(*i, r)$(r, to). (77) 

Since t \ is arbitrary, it can be replaced by t ; Eq. (77) therefore implies Eq. (65). 
Property 2 $(f, to) is invertible and 


[0(f, fo)] 1 = *ft), t). 


(78) 



Chapter 8 Linear Differential Equations 


Proof This resuit is obvious if A(t) is a constant matrix. We know from 
Section 7.8 that the inverse of e Al is e ~ At , hence, 

[♦( f . fo)] -1 = [ e A(r - r °)] _1 = e - A 0 -' o ) 

= e A(?0_?) = <î>(to, t). 

In order to prove Property 2 for any A fi), we note that (65) is valid for any t , hence 
it must be valid for t — to. Thus 

$fio, r)Q(r, to) = Q(to, to). 


It follows from (60) that 


*(fo, t)^(r, ?o) = I 

Thus, from the définition of the inverse, we hâve 

[4»(r, ro)]” 1 = $(f 0 , r) 


which implies (78). 


□ 


Example 3 Find the inverse of 



Solution This matrix is a transition matrix. (See Problem 1.) Hence using (78) 
we find the inverse to be 



Here we hâve used the identifies sin(— 0) = — sin 0 and cos(— 0) = cos 0. ■ 


8.5 Solutions of Systems — General Case 


Properties 1 and 2 enable us to prove Theorem 2, namely, that the solution of 

x(t) = A (t)x(t) +f(0, 

X (to) = c 


is 


x(t) = <P(t, to)c -h f 0>(t, s)f(s)ds. 

JtQ 



Using Property 1, we hâve that &(t,s) — &(t, to)&(to,s); hence, (79) may be 
rewritten as 

x(t) = to)c + &(t, to) f &(to, s)f(s)ds. (80) 

JtQ 


Now 


x(*o) = $(fo, *o)c + $(fo, ^o) *(to, s)f(s)ds 



= le + 10 = c. 

Thus, the initial condition is satisfied by (80). To show that the differential équation 
is also satisfied, we differentiate (80) and obtain 


dx(t) d 

dt dt 


t 0 )c + $(f, to) I *(to, s)f(s)ds 

to 


d ~|f d 

-&(t,to) c+ —Q(t 
dt dt 


■ ' o) ] L 


*(f 0 , s)f(s)ds 


+ Q(t, to) \j t j' ®(to, s)f (s)ds 


A(t)&(t, t 0 )c + A(t)Q(t, t 0 ) / $Oo, s)f(s)ds 

Jt 0 

+ *(t, to)*(to, t)f(t) 


= A (t) 


<b(t, fo)c + $(f, to) f &(to,s)f(s)ds 

Jt o 

-1 


+ *a<o)*- 1 (ï,/o)f(0. 


The quantity inside the bracket is given by (80) to be x(f); hence 


d\(t ) 
dt 


= A (t)x(t) + f(t). 
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We conclude this section with one final property of the transition matrix, the 
proof of which is beyond the scope of this book. 


Property 3 


det <1 >(f, fo) = exp 




Since the exponential is never zéro, (81) establishes that det 4>0, ^o) 7 ^ 0, hence, 
we hâve an alternate proof that 4>(Y, ^o) is invertihle. 


Problem 8.5 

1. Use (59) and (60) to show that 


*(t, t 0 ) = 


cos 


t 2 -t 2 
1 *0 


sin 


t 2 -t 2 

1 fi) 


— sin 


t 2 -t 2 

1 fi) 


cos 


t 2 -t 2 
1 (o 

2 


is a transition matrix for 

x = ty , 
y = —tx. 

2. As a generalization of Problem 1, use (59) and (60) to show that 


$(Mo) = 


cos / g(s)ds sin / g(s)ds 
Jto Jto 

sin / g(s)ds cos / g(s)ds 

Jto Jto 


is a transition matrix for 

x = g(t)y , 
y = -g(0x. 

3 . Prove Lemma 1. (Hint: Consider the product B (Oc where 



"1" 


"0" 


0 


1 

first, c = 

0 

, second, c = 

0 


_0_ 


_0_ 


etc.) 



8.6 Final Comments 



If 4>(ï, to) is a transition matrix, prove that 




f 

UtQ 


i — 1 


4>(^i, s)$ T (ti, s)ds 


*(t l9 to) 


f 

Ut 0 


i — 1 




8.6 Final Comments on Chapter 8 

We now prove that there exists a unique matrix $(7, to) having properties (59) 
and (60). 

Define n-dimensional unit vectors ei, e 2 , . . . , e w by 


Thus, 



~r 


"0" 


"0" 


"0" 

0 


1 


0 


0 

0 


0 


1 


0 

0 

» e 2 = 

0 

,e 3 = 

0 

, . . . , £ n — 

0 

_o_ 


_0_ 


_0_ 


_1_ 


[ei e 2 e 3 



0 0 
1 0 
0 0 



0 0 0 ••• 1 


Consider the homogeneous Systems given by 

x(r) = A (t)x(t) 

x(*o) = *j U — 1 , 2 ,..., n), 





where A (t) and to are taken from (58). For each j(j — 1,2 , . . . , n), Theorem 1 
of Section 8.1 guarantees the existence of a unique solution of (84); dénoté this 
solution by xj (t ) . Thus, xi (t) solves the System 


xi(0 = A(t)xi(t) 

xi(^o) = ei, 


( 85 ) 
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x 2 (0 satisfies the System 

x 2 (r) = A(0 x 2 (0 

x 2 (r 0 ) = e 2 , (86) 

and x n (r) satisfies the System 

i/i(0 = A(0x„(0 

x»fro) = e n , (87) 


Define the matrix 


= [xi(0 x 2 (0 ••• x n (0L 


Th en 


^o) = [Xi (r 0 ) X2(^o) • • • Xn(to)] 

= [ei e 2 ••• e„] {from (85)— (87) 
= I {from (83) 


and 


d&(t, tp) 
dt 


d 

— [xi(0 x 2 (0 ••• x„(0] 
dt 


= l±l(t) x 2 (0 ... x n (t)] 

= [A(0xi(0 A(0 x 2 (0 ... A(0xn(0] {from (85)— (87) 
= A(0[xi(0 x 2 (0 ••• x n (t)] 




Thus $(f, to), as defined above, is a matrix that satisfies (59) and (60). Since this 
0(f, to) always exists, it follows that there will always exist a matrix that satisfies 
these équations. 

It only remains to be shown that 4>(r, to) is unique. Let ty(t, to) be any matrix 
satisfying (59) and (60). Then the yth column ty(t, to) must satisfy the initial-valued 
problem given by (84). However, the solution to (84) is unique by Theorem 1 of 
Section 8.1, hence, the yth column of ty(t, to) must be xy(r). Thus, 

to) = [xi(0 x 2 (0 • • • x n (t)] = $(t, t 0 ). 

From this équation, it follows that the transition matrix is unique. 



Probability and Markov 
Chains 


9.1 Probability: An Informai Approach 

Our approach to probability will be very basic in this section; we will be more 
formai in the next section. 

We begin by considering a set ; recall that a set can be thought of as a collection 
of objects. For example, consider a deck of regular playing cards, consisting of 52 
cards. This set will be called the sample space or the universe. Now suppose we 
shuffle the deck a number of times and, at random, pick out a card. Assume that 
the card is the King of Diamonds. The action of selecting this card is called an 
event. And we might ask the following question: “How likely are we to pick out 
the King of Diamonds?” 

Before attempting to answer this question, let us consider the following: 

• How many times hâve we shuffled the deck? 

• What do we mean by random? 

• What do we mean by likely? 

The answers to these three “simple” questions touch on a number of advanced 
mathematical concepts and can even go into philosophical areas. For our purposes, 
we will, by and large, appeal to our intuition when quantifying certain concepts 
beyond the scope of this section. 

However, we can give a reasonable answer to our original question. We note 
that the size of our sample space (the deck of cards) is 52. We also observe that 
there is only one way to draw a King of Diamonds, since there is only one King of 
Diamonds in the deck; hence, the size of the desired event is 1. So we make the 
following statement, which should seem plausible to the reader: we will say that 
the probability ofthe desired event is, simply, 1/52. 
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Using mathematical notation, we can let S be the set that represents the 
sample space and let E be the set that represents the desired event. Since the 
number of objects (size) in any set is called the cardinal number , we can write 
N (S) = 52 and N(E ) = 1, to represent the cardinal number of each set. So we now 
write 


P(E) = 


N(E) 
N(S ) 


1 

52 



to dénoté the probability of event E. 

What does this mean? It does not mean that, should we make exactly 52 draw- 
ings of a card— returning it to the deck after each drawing and reshuffling the deck 
each time— we would draw the King of Diamonds exactly once. (Try it!). 

A better interprétation is that over a very large number of trials, the proportion 
of times for which a King of Diamonds would be drawn, would get closer and closer 
to 1 out of 52. 

Continuing with this example, the probability of drawing a Spade (event F) is 
one-fourth, since there are 13 Spades in the deck, 


P(F) = 


N(F ) _ 13 
N(S) ~ 52 


1 

4* 



Another example would be to consider a fair die; let’s call it D. Since there are 
six faces, there are six equally likely outcomes (1, 2, 3, 4, 5, or 6) for every roll of 
the die, N(D) — 6. If the event G is to roll a “3,” then 


P(G) = 


N(G) 
N(D ) 


1 

6 ' 



Experiment by rolling a die “many” times. You will find that the proportion 
of times a “3” occurs is close to one-sixth. In fact, if this is not the case, the die is 
most probably “not fair”. 


Remark 1 From this example it is clear that the probability of any of the six 
outcomes is one-sixth. Note, too, that the sum of the six probabilities is 1. Also, 
the probability of rolling a “7” is zéro , simply because there are no 7s on the any 
of the six faces of the die. 


Because of the above examples, it is most natural to think of probability as 
a number. This number will always be between 0 and 1. We say that an event is 
certain if the probability is 1, and that it is impossible if the probability is 0. Most 
probabilities will be strictly between 0 and 1. 

To compute the number we call the probability of an event , we will adopt the 
following convention. We will divide the number ofways the desired event can occur 
by the total number of possible outcomes. We always assume that each member of 
the sample space is “just as likely ” to occur as any other member. We call this a 
relative frequency approach. 
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Example 1 Consider a fair die. Find the probability of rolling a number that is 
a perfect square. 

As before, the size of the sample space, D, is 6. The number of ways a perfect 
square can occur is two: only “1” or “4” are perfect squares out of the first six 
positive integers. Therefore, the desired probability is 


P(K) = 


N(K) 
N(D ) 




Example 2 Consider a pair of fair dice. What is the probability of rolling a “7”? 

To solve this problem, we first hâve to find the cardinal number of the sample 
space, R. To do this, it may be helpful to consider the dice as composed of one red 
die and one green die, and to think of a “roll” as tossing the red die first, followed 
by the green die. Then N(R ) = 36, because there are 36 possible outcomes. To see 
this, consider Figure 9.1 below. Here, the first column represents the outcome of 
the red die, the first row represents the outcome of the green die and the body is 
the sum of the two dice— the actual number obtained by the dice roll. 

Notice, too, that if we label a “7” roll event Z, then N (Z) = 6, because there 
are six distinct ways of rolling a “7”; again, see Figure 9.1 below. 

So our answer is 


F(Z) = 


N(Z) 
N (R) 




Example 3 Suppose a random number generator generates numbers ranging 
from 1 through 1000. Find the probability that a given number is divisible by 5. 

Elementary Number Theory teaches that for a number to be divisible by 5, the 
number must end in either a “5” or a “0”. By sheer counting, we know that there 
are 200 numbers between 1 and 1000 that satisfy this condition. Therefore, the 
required probability is ■ 
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Figure 9.1 
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In the next section we will give some rules that pertain to probabilities, and 

investigate the meaning of probability more fully. 

Problems 9.1 

1. Find the sample space, its cardinal number and the probability of the desired 

event for each of the following scénarios: 

a) Pick a letter, at random, out of the English alphabet, 
choosing a vowel. 

b) Pick a date, at random, for the Calendar Year 2008. 
choosing December 7th. 

c) Pick a U.S. President, at random, from a list of ail the presidents. Desired 
Event: choosing Abraham Lincoln. 

d) Pick a U.S. President, at random, from a list of ail the presidents. Desired 
Event: choosing Grover Cleveland. 

e) Pick a card, at random, from a well shuffled deck of regular playing cards. 
Desired Event: choosing the Ace of Spades. 

f) Pick a card, at random, from a well shuffled deck of Pinochle playing cards. 
Desired Event: choosing the Ace of Spades. 

g) Roll a pair of fair dice. Desired Event: getting a roll of “2” (Snake Eyes). 

h) Roll a pair of fair dice. Desired Event: getting a roll of “12” (Box Cars). 

i) Roll a pair of fair dice. Desired Event: getting a roll of “8”. 

j) Roll a pair of fair dice. Desired Event: getting a roll of “11”. 

k) Roll a pair of fair dice. Desired Event: getting a roll of an even number. 

l) Roll a pair of fair dice. Desired Event: getting a roll of a number that is a 

perfect square. 

m) Roll a pair of fair dice. Desired Event: getting a roll of a number that is a 
perfect cube. 

n) Roll a pair of fair dice. Desired Event: getting a roll of a number that is a 
multiple of 3. 

o) Roll a pair of fair dice. Desired Event: getting a roll of a number that is 
divisible by 3. 

p) Roll a pair of fair dice. Desired Event: getting a roll of “13”. 

2. Suppose we were to roll three fair dice: a red one first, followed by a white die, 

followed by a blue die. Describe the sample space and find its cardinal number. 

3. Suppose the probability for event A is known to be 0.4. Find the cardinal 

number of the sample space if N(A) = 36. 


Desired Event: 
Desired Event: 
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4. Suppose the probability for event B is known to be 0.65. Find the cardinal 
number of B , if the cardinal number of the sample space, 5, is N(S) = 3000. 


9.2 Some Laws of Probability 


In this section we will continue our discussion of probability from a more 
theoretical and formai perspective. 

Recall that the probability of event A, given a sample space 5, is given by 


P(A) = 


N(A) 
N(S )’ 



where the numerator and denominator are the respective cardinal numbers of A 
and S. 

We will now give a number of définitions and rules which we will follow regard- 
ing the computations of probabilities. This list “formalizes” our approach. We note 
that the reader can find the mathematical justification in any number of sources 
devoted to more advanced treatments of this topic. We assume that A, B, C . . . are 
any events and that S is the sample space. We also use <£ to dénoté an impossible 
event. 


• P(O) = 0; that is, the probability of an impossible event is zéro. 

• P(S) = 1; that is, the probability of a certain event is one. 

• For any two events, A and B , then 

P(A U B) = P(A) + P(B) - P(A H B). (7) 

Remark 1 Here we use both the union (U) and intersection (n) notation from 
set theory. For this rule, we subtract off the probability of the “common” even in 
order not to “count it twice”. For example, if A is the event of drawing a King 
from a deck of regular playing cards, and B is the event of drawing a Diamond, 
clearly the King of Diamonds is both a King and a Diamond. Since there are 4 
Kings in the deck, the probability of drawing a King is And since there are 13 

Diamonds in the deck, the probability of drawing a Diamond is Since there 
is only one King of Diamonds in the deck, the probability of drawing this card is 
clearly We note that 


4 13 1 _ 16 

52 + 52 ~~ 52 ~~ 52’ ^ ’ 

which is the probability of drawing a King or a Diamond, because the deck contains 
only 16 Kings or Diamonds. 

• If A and B are disjoint events, then 


P(A U B) = P(A) + P(B). 


( 9 ) 
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Remark 2 Two events are disjoint if they are mutually exclusive; that is, they 
cannot happen simultaneously. For example, the events of drawing a King from a 
deck of regular playing cards and, at the same time, drawing a Queen are disjoint 
events. In this case, we merely add the individual probabilities. Note, also, that 
since A and B are disjoint, we can write A fl B — (h; hence, P(A n B) = P(O) = 0. 
The reader will also see that équation (9) is merely a spécial case of équation (7). 

• Consider event A; if A c represents the complément of A, then 

P(A C ) = 1 - P(A). (10) 

Remark 3 This follows from the fact that either an event occurs or it doesn’t. 
Therefore, P(A U A c ) — 1; but since these two events are disjoint, P(A U A c ) — 
P(A) + P(A C ) — 1. Equation (10) follows directly. 

For example, if the probability of rolling a “3” on a fair die is g, then the 
probability of not rolling a “3” is 1 — \ 

In the next section we will introduce the idea of independent events , along with 
associated concepts. For the rest of this section, we give a number of examples 
regarding the above rules of probability. 


Example 1 Given a pair of fair dice, find the probability of rolling a “3” or a “4”. 

Since these events are disjoint, we use Equation (9) and refer to Figure 9.1 and 
obtain the desired probability: ^ -h ^ ■ 


Example 2 Given a pair of fair dice, find the probability of not rolling a “3” or 
a “4”. 

From the previous example, we know that the probability of rolling a “3” or 
a “4” is therefore, using Equation (10), we find that the probability of the 

complementary event is: 1 — ^ ■ 


Remark 4 Note that we could hâve computed this probability directly by count- 
ing the number of ways —31 —in which the rolls 2, 5, 6, 7, 8, 9, 10, 11 or 12 can occur. 
However, using Equation (10) is the preferred method because it is quicker. 


Example 3 Pick a card at random out of a well shuffled deck of regular playing 
cards. What is the probability of drawing a picture card (that is, a King, Queen, or 
Jack)? 

Since there are four suits (Spades, Hearts, Clubs, and Diamonds), and there 
are three picture cards for each suit, the desired event can occur 12 ways; these 
can be thought of as 12 disjoint events. Hence, the required probability is ■ 
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Example 4 Pick a card at random out of a well shuffled deck of regular playing 
cards. Find the probability of drawing a red card or a picture card. 

We know there are 12 picture cards as well as 26 red cards (Hearts or Dia- 
monds). But our events are not disjoint, since six of the picture cards are 

red. Therefore, we apply Equation (7) and compute the desired probability as 

12 _i_ 26 6_ 32 m 

52 ' 52 52 — 52- 


Example 5 Suppose events A and P are not disjoint. Find P(A fl B), if it is given 
that P(A) = 0.4, P(B) = 0.3, and P(A U B) = 0.6. 

Recall Equation (7): P(A U B) — P(A) + P(P) — P(A H B). Therefore, 0.6 = 
0.4 + 0.3 — P(A D B). Therefore, P(A fl B) — 0.1. ■ 


Example 6 Extend formula (7) for three non-disjoint events. That is, consider 
P(A U BUC). 

By using parenthèses to group two events, we hâve the following équation 
below: 

P(A U B U C) = P((A U B) U C) = P(A U B) + P(Ç) - P((A U B)) fl C). (11) 

From Set Theory, we know that the last term of (11) can be written as 
P((A U B) fl C) = P((A H C) U (B n C)). Hence, applying (7) to (11) yields 

P(A nc) + P(B n C) - P((A n C) n (P n c)). (12) 

But the last term of (12) is équivalent to P(A n BUC). After applying (7) to 
the P(A U B) term in (11), we hâve 

P(A U B U C) = P(A) + P(B) + P(C) - P(A fl B) 

- [P(A nc) + P(B n C) - P((A n C) n (P n c))]. (13) 

This simplifies to: 

P(A U P U C) = P(A) + P(B) + P(C ) - P(A HP) - P(A fl C) 

- p(B n C) + P(A n p n C). (14) 


Remark 5 Equation (14) can be extended for any finite number of events, and 
it holds even if some events are pairwise disjoint. For example, if events A and P 
are disjoint, we merely substitute P(A n P) = 0 into (14). 
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Problems 9.2 

1. Pick a card at random from a well shuffled deck of regular playing cards. Find 
the probabilities of: 

a) Picking an Ace or a King. 

b) Picking an Ace or a picture card. 

c) Picking an Ace or a black card. 

d) Picking the Four of Diamonds or the Six of Clubs. 

e) Picking a red card or a Deuce. 

f) Picking a Heart or a Spade. 

g) Not choosing a Diamond. 

h) Not choosing a Queen. 

i) Not choosing an Ace or a Spade. 

2. Roll a pair of fair dice. Find the probabilities of: 

a) Getting an odd number. 

b) Rolling a prime number. 

c) Rolling a number divisible by four. 

d) Not rolling a “7”. 

e) Not rolling a “6” or an “8”. 

f) Not rolling a “1”. 

3. See Problem 2 of Section 9.1. Roll the three dice. Find the probabilities of: 

a) Getting an odd number. 

b) Rolling a “3”. 

c) Rolling an “18”. 

d) Rolling a “4”. 

e) Rolling a “17”. 

f) Rolling a “25”. 

g) Not rolling a “4”. 

h) Not rolling a “5”. 

i) Not rolling a “6”. 

4 . Consider events A and B. Given P(A) = .7 and P(B) = .2, find the probability 
of “A or if P(A n B) = .15. 
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5. Suppose events A and B are equally likely. Find their probabilities if P(A U B) = 
.46 and P(A = .34. 

6. Extend Equation (14) for any four events A, B, C, and D. 


9.3 Bernoulli Trials and Combinatorics 

In the previous section, we considered single events. For example, rolling dice once 
or drawing one card out of a deck. In this section we consider multiple events which 
neither affect nor are affected by preceding or succeeding events. 

For this section, we will consider events with only two outcomes. For example, 
flipping a coin, which can resuit in only “heads” or “tails”. The coin cannot land on 
its edge. We do not insist that the probability of a “head” equals the probability 
of a “tail”, but we will assume that the probabilities remain constant. Each one of 
the “flips” will be called a Bernoulli trial , in honor of Jakob Bernoulli (1654-1705). 

Remark 1 A closely related underlying mathematical structure for these trials 
is known as a Binomial Distribution. 

Remark 2 The Bernoulli family had a number of great mathematicians and 
scientists spanning several générations. This family is to mathematics what the 
Bach family is to music. 

As we hâve indicated, we will assume that the events are independent. Hence 
the probabilities are unaffected at ail times. So, if we tossed a coin 10 times in a 
row, each of the tosses would be called a Bernoulli trial , and the probability of 
getting a head on each toss would remain constant. 

We will assume the following rule. If two events, A and B , are independent, 
then the probability of “A and B ” or the probability of “A followed by B ” is given 
by: 


P(AB) = P(A H B) = P(A)P(B). 



Notice that we use the intersection (fl) notation. This simple rule is called the 
multiplication rule. 

Remark 3 The reader must be careful not to confuse disjoint events with inde- 
pendent events. The former means that “nothing is in common” or that the events 
“cannot happen simultaneously”. The latter means that the probabilities do not 
influence one another. Often, but not always, independent events are sequential; 
like flipping a coin 10 times in a row. 

It is clear that probabilities dépend on counting, as in determining the size of 
the sample space. We assume the following resuit from an area of mathematics 
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known as Combinatorics : 

• The number of ways we can choose k objects from a given collection of n objects 
is given by: 


Remark 4 This is équivalent to determining the number of subsets ofsize k given 
a set ofsize n , where k < n. 

Remark 5 We saw “factorials” in Chapter Seven. Recall that 31, for example, 
is read “three factorial” and it is evaluated 3 • 2 • 1 = 6. Hence, “« — factorial” is 
given by n\ — n(n — l)(n — 2) • • • 3 • 2 • 1. By convention, we define 0! = 1. Finally, 
we only consider cases where n is a non-negative integer. 

Remark 6 For these “number of ways”, we are not concerned about the order 
of sélection. We are merely interested in the number of combinations (as opposed 
to the number of permutations). 

We will provide the reader with a number of examples which illustrate the 
salient points of this section. 


n\ 


k\(n-k)\ 


(16) 


Example 1 Evaluate (j)- Using (16) we see that 
Since 5! = 120, 2! = 2 and 3! = 6 , ( 2 ) = TT = 10 - 


5! 


5! 


2! (5 — 2) ! — 2!3! * 


Example 2 Given a committee of five people, in how many ways can a sub- 
committee of size two be formed? The number is precisely what we computed 
in the previous example: 10. The reader can verify this as follows. Suppose the 
people are designated: A, B, C, D , and E. Then, the 10 sub-committees of size two 
are given by: AB, AC, AD, AE, BC, BD, BE, CD, CE, and DE. ■ 


Example 3 Given a committee of five people, in how many ways can a sub- 
committee of size three be formed? We can use formula (16) to compute the 
answer, however the answer must be 10. This is because a sub-committee of 3 is 
the complément of a sub-committee or two. That is, consider the three people not 
on a particular sub-committee of two, as constituting a sub-committee of three. 
For example, if A and B are on one sub-committee of size two, put C, D, and E on 
a sub-committee of size three. Clearly there are 10 such pairings. ■ 


Example 4 Suppose we flip a fair coin twice. What is the probability of getting 
exactly one “head”? 
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Let H represent getting a “head” and T represent getting a “tail”. Since the coin 
is fair, P(H) — P(T) — \. The only way we can obtain exactly one head in two tosses 
is if the order of outcomes is either HT or TH. Note that the events are disjoint 
or mutually exclusive ; that is, we cannot get these two outcomes at the same time. 
Hence, Equation (9) will corne into play. And because of the independence of the 
coin flips (each toss is a Bernoulli trial), we will use Equation (15) to détermine 
the probability of obtaining HT and TH. 

Therefore, the probability of getting exactly one H is equal to 

P(HT U TH) = P(HT) + P(TH) = P(H)P(T) + P(T)P(H) 

_ i i i i _ i (17) 

~ 2 .' 2 + 2 .' 2 ~ 2 ' 


Remark 7 Note that (17) could hâve been obtained by finding the probability of 
HT —in that order— and then multiplying it by the number o fûmes ( combinations ) 
we could get exactly one H in two tosses. 


Example 5 Now, suppose we flip a fair coin 10 times. What is the probability of 
getting exactly one “head”? 

Suppose we get the H on the first toss. Then the probability of getting 

HTTTTTTTTT —in that order— is equal » because the tosses 

are ail independent. Note that if the H occurs “in the second slot”, the probability 
is also ïfâÂ- In fact, we get the same number for ail 10 possible “slots”. Hence the 
final answer to our question is ^4 • ® 


Remark 8 Note that the previous example could hâve be answered by the 
following computation 


/m/n/n 9 - 10 

v 1 ) w = ÏÔ24' 

Here, the first factor gives the number of ways we can get exactly one H 
in 10 tosses; this is where mutual exclusivity cornes in. The second factor is the 
probability of getting one H , and the third factor is the probability of getting nine 
T s; the independence factor is employed here. 



Example 6 Suppose we flip a fair coin 10 times. Find the probability of getting 
exactly five H s. 
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Since there are 



probability is given by 


= 252 ways of getting five H s in 10 tosses, the desired 



Example 7 Suppose we flip an unfair coin 10 times. If P(H) — .3 and the P(T) — 
.7, what is the probability of getting exactly five H s? 

As we learned from the previous problem, there are 252 ways of getting exactly 
five H s in 10 tosses. Hence, our desired probability is given by 

V.3) 5 (.7) 5 « 0.103. ■ 



Remark 9 A calculator is useful for numerical computations. We will address 
the issues of calculations and technology in both Section 9.5 and in the Appendix. 
Note, too, that individual probabilities, P(H) and P(T) must add to 1, and that 
the exponents in the formula must add to the total number of tosses; in this 
case, 10. 


Example 8 Consider the previous example. Find the probability of getting at 
le as t five H s. 

Note that at leastfive H s means exactly five H s plus exactly six H s plus ... etc. 
Note, also, that exactly five H s and exactly six H s are disjoint, so we will use 
Equation (9). Therefore the desired probability is given by: 

(g 0 ) (-3) 5 (.7) 5 + (.3) 6 (.7) 4 + (,3) 7 (.7) 3 + (.3) 8 (.7) 2 

+ (9°) (-^V) 1 + Q (•3) 10 (.7) 10 « 0.150. (20) 


Example 9 Consider the previous examples. Find the probability of getting at 
least o ne H. 



9.3 Bernoulli Trials and Combinatorics 


309 


While we could follow the approach in Example 9, there is a simpler way to 
answer this question. If we realize that the complément of the desired event is 
getting no H s in 10 tosses, we can apply Equation (10). That is, the probability of 
getting at least one H is equal to 




(.3)°(.7) 10 » 0.972. 



We summarize the results for the probability in this section as follows: 

• Given n successive Bernoulli trials, the probability of getting exactly k 
successes, where k < n, is equal to 


k\(n — k)\ 


p k ( i - p) n ~ k 



where the probability of a success is /?, and the probability of a failure is (1 — p). 


Problems 9.3 

1. Evaluate the following: 



2. How many different nine-player line-ups can the New York Yankees produce 
if there are 25 players on the roster and every player can play every position? 

3. Suppose 15 women comprised a club, and a committee of 6 members was 
needed. How many different committees would be possible? 

4. Toss a fair die eight times. Find the probability of: 

a) Rolling exactly one “5”. 

b) Rolling exactly three “5s”. 

c) Rolling at least three “5s”. 

d) Rolling at most three “5s”. 

e) Rolling at least one “5”. 

5. Suppose event A has a probability of occurring equal to .65. Without evalu- 
ating the expressions, find the following probabilities given 500 independent 
Bernoulli trials. 
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a) Event A occurs 123 times. 

b) Event A occurs 485 times. 

c) Event A occurs at least 497 times. 

d) Event A occurs at most 497 times. 

e) Event A occurs any non-zero multiple of 100 times. 

6. An urn contains 10 golf balls, three of which are white, with the remaining 
seven painted orange. A blindfolded golfer picks a golf bail from the urn, and 
then replaces it. The process is repeated nine times, making a total of 10 trials. 
What is the probability of the golfer picking a white golf bail exactly three 
times? 

7. An urn contains 10 golf balls colored as follows: three are white; two are green; 
one is red; four are orange. A blindfolded golfer picks a golf bail from the urn, 
and then replaces it. The process is repeated nine times, making a total of 10 
trials. What is the probability of the golfer picking a white golf bail exactly 
three times? 


9.4 Modeling with Markov Chains: An Introduction 

In Chapter 1 and Chapter 6, we mentioned the concept of Markov chains. We 
return to this idea, further formalizing it from the perspective of probability. 
Consider the following example, which we will call the Moving Situation. 


Example 1 Suppose we hâve two families. Family (1) lives in State A and Family 
(2) lives in State B. Let us further assume that the matrix 





represents the following probabilities. The element in the first row and first column 
represents the probability of Family (1) originally residing in State A remaining 
in State A, while the element in the first row and second column represents the 
probability of starting in State A and then moving to State B. Note that these two 
probabilities add to one. 

Similarly, let the element in the second row and first column, represent the 
probability of Family (2) starting in State B and moving to State A, while the ele- 
ment in the second row and second column, represents the probability of starting 
in State B and remaining in State B. Here, too, these two probabilities add to one. 

Note that we can consider the process as “time conditioned” in the sense that 
there is a présent and a future (for example, one year from the présent). 

Such a matrix is called a transition matrix and the éléments are called 
transitional probabilities. 
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Let us consider the matrix in (23) and let us compute P 2 . We find that 



.76 .24 
.72 .28 ' 



What does P 2 represent? To answer this question, let us ask another question: 
From the perspective of Family (1), what is the probability of being in State A after 
two years? 

There are two ways Family (1) can be in State A after two years: 

• Scénario 1: Either the family stayed two years in a row. 

• Scénario 2: The family moved to State B after one year and then moved back 

to State A after the second year. 

The probability of the first scénario is .7(.7) = .49, because these events can be 
considered independent. The probability of the second is .3 (.7) = .27. 

Because these events are disjoint , we add the probabilities to get .76. 

Note that this is the element in the first row and first column of P 2 . 

By similar analyses we find that P 2 is indeed the transitional matrix of our 
Moving Situation after two time periods. 

Matrix P is the transition matrix for a Markov chain. The sum of the proba- 
bilities of each row must add to one, and by the very nature of the process, the 
matrix must be square. We assume that at any time each object is in one and only 
one State (although different objects can be in the same State). We also assume 
that the probabilities remain constant over the given time period. ■ 


Remark 1 The notation p^ n \j is used to signify the transitional probability of 
moving from State i to State j over n time periods. 


Example 2 Suppose Moe, Curly, and Larry live in the same neighborhood. Let 
the transition matrix 



.1 

.3 

.1 


.2 

.2 

.1 



represent the respective probabilities of Moe, Curly, and Larry staying at home 
on Monday and either visiting one of their two neighbors or staying home on 
Tuesday. We ask the following questions regarding Thursday: 

a) What is the probability of Moe going to visit Larry at his home, p (3) 13 ? 

b) What is the probability of Curly being at his own home, p (3) 2 2 ? 
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To answer both of these questions, we must compute P 3 because three time 
periods would hâve elapsed. We find that 


.694 

.124 

.182 

.124 

.132 

.182 

.695 

.124 

.181 



So, our answers are as follows: a) the probability is .182, the entry in the first 
row and third column; b) the probability is .132, the entry in the second row and 
second column. ■ 


Example 3 Consider the transitional matrix K which represents respective 
probabilities of Republicans, Democrats, and Independents either remaining 
within their political parties or changing their political parties over a two-year 
period: 



.7 

.15 

.3 


.1 .2 
.75 .1 . 
.2 .5 



What is the probability of a Republican becoming an Independent after four years? 
And what is the probability of a Democrat becoming a Republican after four 
years? 

Both of these questions require two time periods; hence we need p (2) 13 and 
p (2) 2 i which can be obtained from K 2 below: 



' .565 

.185 

.25 

— 

.2475 

.5975 

.155 


.39 

.28 

.33 

___ 

.2475. 




(28) 


We close this discussion with the observation that sometimes transitional 
matrices hâve spécial properties. For example, consider the matrix 





We find that if we raise this matrix to the 9th and lOth powers, our resuit is the 
same. That is, 


A 9 = A 10 


.375 .625 
.375 .625 ' 



and the same resuit occurs for any higher power of A. This absorbing quality 
implies that “sooner or later” the transitional probabilities will stabilize. 
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Markov processes are used in many areas including decision theory, économies 
and political science. ■ 


Problems 9.4 


1. Why are the following matrices not transitional matrices? 

.1 .5 .4 

.2 .6 .2 


a) 


0 1 
1 2 


b) 


.6 .5 
.4 .5 


• JL . / 

c) I 1 0 0 

non 


d) 


2. Consider the following transitional matrices. Construct scénarios for which 
these matrices might represent the transitional probabilities: 


a) 

".5 .5" 

; b) 

".95 

.05' 

; c) 

'.5 

.5" 


n .3 

.02 

,98_ 

.5 

.5 

• 

5 


"1 0 " 
0 .1 


"o r 
1 0 



".1 

.2 .7" 

d) 

; e) 

• 

1 

f) 

.5 

.25 .25 







.3 

.3 .4 


3. Consider the c) and d) matrices in the previous problem; show that these 
matrices are “absorbing” matrices. 


4. Consider the e) matrix in Problem (2). Raise the matrix to the powers of 2, 3, 
4, 5, 6, 7, and 8. What do you notice? Can you construct a scénario for which 
this transitional matrix could be a model? 


5. Consider the following transitional matrix: 
and p {3) 22 . 


.6 

.1 


À 

.9 


. Find /? (2) n , p w 21 , 


( 2 ) 


,( 3 ) 


6. Consider a game called Red-Blue. The rules State that after one “turn” Red can 
become Blue or remain Red. The same is true with Blue. Suppose you make a 
bet that after five turns, Red will be in the Red category. You are told that the 
following probabilities are valid: 


• Given Red, the probability of remaining Red after one turn is .7 

• Given Red, the probability of going to Blue is .3 

• Given Blue, the probability of remaining Blue is .6 

• Given Blue, the probability of going to Red is .4 

a) Give the transition matrix. 

b) What is the probability of you winning your bet? 

c) Does the probability increase, decrease or stay the same if you bet six turns 
instead of five? 
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9.5 Final Comments on Chapter 9 

Probability is a fascinating area. For numbers that necessarily range between 0 
and 1, inclusively, a lot can happen. 

When using probability, we must understand exactly what is being asked and 
give précisé answers, without misrepresenting our conclusions. Concepts such as 
randomness and independence must be présent before certain laws can be applied. 
While the mathematical underpinnings are rock solid, probabilities generally deal 
with “trends” and “likelihoods”. 

Regarding Bernoulli trials, if the number of experiments is large , the calcu- 
lations can be overwhelming. In these cases, the use of computers and other 
technological aids is essential. From a theoretical perspective, there is a very good 
approximation that can be employed, known as the Normal Approximation to the 
Binomial Distribution. This technique is explored in basic courses on probability 
and statistics. 

One final point: With the exception of the section on Markov chains, ail the 
probabilities in Chapter 9 were theoretically assigned. That is, we made assump- 
tions, applied définitions and then made our computations. For example, if a die 
was fair, then we assigned a probability of ^ to the event of rolling a “3”, based on 
our définition, which dealt with relative frequency. 

However, there are many times when probabilities are obtained by observation 
and empirical evidence. For example, the greatest baseball player of ail time, Babe 
Ruth, had a lifetime batting average of .342. Since batting average is defined as 
successful hits divided by total at-bats, we can interpret this as Ruth getting 342 
hits for every 1000 at-bats over a long period oftime. 

There are many other occurrences of empirical probabilities in research areas 
such as medicine, psychology, économies, and sociology, to name but a few. 




Real Inner Products and 
Least-Square 


10.1 Introduction 

To any two vectors x and y of the same dimension having real components (as 
distinct from complex components), we associate a scalar called the inner product, 
denoted as (x, y), by multiplying together the corresponding éléments of x and y, 
and then summing the results. Students already familiar with the dot product of 
two- and three-dimensional vectors will undoubtedly recognize the inner product 
as an extension of the dot product to real vectors of ail dimensions. 


Example 1 Find (x, y) if 



Solution (x, y) = 1(4) + 2(-5) + 3(6) = 12. ■ 


Example 2 Find (u, v) if u = [20 —4 30 10] and v = [10 —5 —8 —6]. 

Solution (u, v) = 20(10) + (— 4)(— 5) + 30(-8) + 10(-6) = -80. ■ 

It folio ws immediately from the définition that the inner product of real vectors 
satisfies the following properties: 

(11) (x, x) is positive if x ^ 0; (x, x) = 0 if and only if x = 0. 

(12) (x, y) = (y, x). 

(13) (Àx, y) = À(x, y), for any real scalar À. 
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(14) (x + z, y) = (x, y) + (z,y). 

(15) (0, y) = 0. 

We will only prove (II) here and leave the proofs of the other properties 
as exercises for the students (see Problems 29 through 32). Let x = 
[x\ X 2 X 3 • • • x n ] be an n-dimensional row vector whose components 
xi, X 2 , * 3 , . . . , x n are ail real. Then, 

(x, x) = ( x \ ) 2 + ( X 2 Ÿ + fe ) 2 + • • * + (%nŸ • 

This sum of squares is zéro if and only if x\ = X 2 = ^3 = • • • = x n — 0, which in 
turn implies x = 0. If any one component is not zéro, that is, if x is not the zéro 
vector, then the sum of squares must be positive. 

The inner product of real vectors is related to the magnitude of a vector as 
defined in Section 1.6. In particular, 

Il x || = y/(X,X). 


Example 3 Find the magnitude of x = [2 —3 —4]. 

Solution (x, x) = 2(2) + (—3) (—3) + (—4) (—4) = 29, so the magnitude of x is 

Il x || = V29. ■ 


The concepts of a normalized vector and a unit vector are identical to the 
définitions given in Section 1.6. A nonzero vector is normalized if it is divided by 
its magnitude. A unit vector is a vector whose magnitude is unity. Thus, if x is any 
nonzero vector, then (l/||x||) x is normalized. Furthermore, 


( 





1 

nr x > 

X II 

(Property 13) 

-X,x) 

(Property 12) 

<x,x) 

(Property 13) 

M 2 = l, 



so a normalized vector is always a unit vector. 
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Problems 10.1 


In Problems 1 through 17, find (a) (x, y) and (b) (x, x) for the given vectors. 


1. x = 


2. x = 


3. x = 


1' 

2 

' 2 ' 

0 


and y = 


and v = 


'3' 

4 


4 

-5 


■5 

7 


3' 
-5 


4. x 

5. x 


and y = 

L ' J L 

[3 14] and y = [7 3]. 

= [-2 -8] and y = [-4 


-7], 


6. x = 


" 2 " 

0 

1 


and y 


T 

2 

4 


7. x = 

"-2 

2 

and 

y = 

"- 4 " 

3 

8. x = 

_- 4 _ 

"-3“ 

-2 

and 

y = 

3_ 

6" 

-4 


5 



-4 


9. x 

10. x 

11. x 

12. x 


= [* 


1 

3 


h] and y = [3 1 i]- 

[1/V2 1/V3 1/V6] and y = [l/V3 3/V2 l] 

[* h y - d y = tè \ è]- 

[10 20 30] and y = [5 -7 3]. 


13. x = 


14. x = 


T 

0 

1 

1 

'i 

2 

1 

2 

1 

2 

1 

2 


and y = 


and y = 


T 

1 

0 

1 


1' 

2 

3 

-4 



r 3 


^ 4" 

15. x = 

5 

-7 

and y = 

-6 

-9 


-8 


8 
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21 . 

22 . 



24. 














x = [5 5 5 5 5] and y = t 1 2 - 3 4 -5]- 

x = [1 1 1 1 1 1] and y = [-3 8 11 -4 7]. 
Normalize y as given in Problem 1. 

Normalize y as given in Problem 2. 

Normalize y as given in Problem 4. 

Normalize y as given in Problem 7. 

Normalize y as given in Problem 8. 

Normalize y as given in Problem 11. 

Normalize y as given in Problem 15. 

Normalize y as given in Problem 16. 

Normalize y as given in Problem 17. 

Find x if (x, a) b = c, where 


a = 

1 

3 

. b = 

"2 

1 

, and c = 

3" 

0 


-1 


1 


-1 


Détermine whether it is possible for two nonzero vectors to hâve an inner 
product that is zéro. 

Prove Property 12. 

Prove Property 13. 

Prove Property 14. 

Prove Property 15. 

Prove that ||x + y || 2 = ||x|| 2 + 2(x, y) + ||y || 2 . 

Prove the parallelogram law : 

ll x + yll 2 + ll x — y II 2 = 2||x|| 2 + 2||y|| 2 . 

Prove that, for any scalar À, 

0 < IIA.X - y|| 2 = À 2 ||x|| 2 - 2k (x, y> + ||y|| 2 . 

(Problem 35 continued) Take k = (x, y)/||x|| 2 and show that 


0 < 
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From this, deduce that 


(x,y r < llx 


2 


and that 


(x,y)| < ||x|| ||y||. 


This last inequality is known as the Cauchy-Schwarz inequality. 

37. Using the results of Problem 33 and the Cauchy-Schwarz inequality, show 
that 


II* + yll 2 < IMI 2 + 2||x|| ||y II + Hy 
= (INI + llyll) 2 - 

From this, deduce that 


x + y|| < l|x|| + llyll- 


38. Détermine whether there exists a relationship between (x, y) and x T y, when 
both x and y are column vectors of identical dimension with real components. 

39. Use the results of Problem 38 to prove that (Ax, y) = (x, A T y), when A, x, 
and y are real matrices of dimensions n x n, n x 1 , and n x 1 , respectively. 

40. A generalization of the inner product for n-dimensional column vectors with 
real components is (x, y ) a = (Ax, Ay) for any real n x n nonsingular matrix 
A. This définition reduces to the usual one when A = I. 


Compute (x, y ) a for the vectors given in Problem 1 when 



2 3 

1 -1 • 


41. Compute (x, y ) a for the vectors given in Problem 6 when 



1 0 

0 1 . 

1 1 



1 -1 1 

0 1 -1 . 

1 1 1 


42. Redo Problem 41 with 
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10.2 Orthonormal Vectors 

Définition 1 Two vectors x and y are orthogonal (or perpendicular) if (x, y) = 0. 
Thus, given the vectors 


X = 

"Il 

1 

> y = 

"-i" 

i 

, z = 

" 1 " 

1 


1 


0 


0 


we see that x is orthogonal to y and y is orthogonal to z since (x, y) = (y, z) = 0; 
but the vectors x and z are not orthogonal since (x, z) = 1 + 1 ^ 0. In particular, 
as a direct conséquence of Property (15) of Section 10.1 we hâve that the zéro 
vector is orthogonal to every vector. 

A set of vectors is called an orthogonal set if each vector in the set is orthogonal 
to every other vector in the set. The set given above is not an orthogonal set since 
z is not orthogonal to x whereas the set given by {x, y, z}, 


x = 

" 1 " 

1 

> y = 

i 

i 

, z = 

1 " 

-1 


1 


-2 


0 


is an orthogonal set because each vector is orthogonal to every other vector. 

Définition 2 A set of vectors is orthonormal if it is an orthogonal set having the 
property that every vector is a unit vector (a vector of magnitude 1). 

The set of vectors 


' 

"1/V2" 


1/V2" 


"0 



1/V2 

1 

— 1/V2 

5 

0 



0 


0 


1 



is an example of an orthonormal set. 

Définition 2 can be simplified if we make use of the Kronecker delta, 8^ 
defined by 



if i = j, 

if i ^ j. 



A set of vectors {xi, X 2 , . . . , x n } is an orthonormal set if and only if 

(x/, Xj) = 8jj for ail i and j , i, j = 1, 2, . . . , n. (2) 

The importance of orthonormal sets is that they are almost équivalent to lin- 
early independent sets. However, since orthonormal sets hâve associated with 
them the additional structure of an inner product, they are often more conve- 
nient. We devote the remaining portion of this section to showing the équivalence 
of these two concepts. The utility of orthonormality will become self-evident in 
later sections. 
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Theorem 1 An orthonormal set of vectors is linearly independent. 

Proof. Let {xi, X 2 , . . . , x rt } be an orthonormal set and consider the vector 
équation 


cixi + c 2 \2 H 1- c n x n = 0 (3) 

where the c/s (j = 1,2 , ,n) are constants. The set of vectors will be linearly 
independent if the only constants that satisfy (3) are c\ = C 2 = • • • = c n = 0. Take 
the inner product of both sides of (3) with xi. Thus, 


(ci xi + c 2 x 2 H h C n x n ,x i) = ( 0 , xi ) . 


Using properties (13), (14), and (15) of Section 10.1, we hâve 


ci(xi,xi) +c 2 (x 2 ,xi) H \-c n (x n ,x i) = 0. 

Finally, noting that (x/, xi) = Su, we obtain c\ — 0. Now taking the inner prod- 
uct of both sides of (3) with x 2 , X 3 , . . . , x n , successively, we obtain c 2 = 0, 
C 3 = 0, . . . , c n = 0. Combining these results, we find that ci = c 2 = • • • c n = 0, 
which implies the theorem. □ 

Theorem 2 For every linearly independent set of vectors {xi, x 2 , . . . , x n }, there 
exists an orthonormal set of vectors {qi, q 2 , . . . , q /7 } such that each q ,{j — 
1 , 2 , . . . , n) is a linear combination ofx\, x 2 , . . . , xj. 

Proof. First define new vectors vi . V 2 , . . . , y„ by 




(X2, yt) 
(yt.yt) 




(X3,yt) 

(yi,yt> 



(X3,y2) 

(y2, y 2) 



and, in general, 



(X;,yt) 
(y*, yk) 


y k (J — 2,3, , n). 



Each y y is a linear combination of xi, X 2 , . . . , x y (j = 1,2, ... , n). Since the x’s are 
linearly independent, and the coefficient of the xj term in (4) is unity, it follows 
that y j is not the zéro vector (see Problem 19). Furthermore, it can be shown that 
the y j terms form an orthogonal set (see Problem 20), hence the only property 



Chapter 1 0 Real Inner Products and Least-Square 


that the y y terms lack in order to be the required set is that their magnitudes may 
not be one. We remedy this situation by defining 



The desired set is {qi, q2, . . . , q„}. 

The process used to construct the q 7 terms is called the Gram-Schmidt 
orthonormalization process. □ 


Example 1 Use the Gram-Schmidt orthonormalization process to construct 
an orthonormal set of vectors from the linearly independent set {xi,X 2 ,X 3 }, 
where 



r l 


0 


1 

XI = 

1 

O h-> 

1 

, x 2 = 

1 

1 

, x 3 = 

0 

1 


Solution 



1 

1 

0 


hence, 


Now (x 2 , yi) = 0(1) + 1(1) + 1(0) = 1, and (yi, yi) = 1(1) + 1(1) + 0(0) 


y2 = x 2 - 


(* 2 ,yi) 

(yi.yi) 


1 

yi = x 2 - ^yi 



- 1/2 
1/2 . 
1 


Then, 

(x 3 , yi) = 1(1) + 0(1) + 1(0) = 1, 

(* 3 . y 2 ) = 1 (— 1 / 2 ) + 0 ( 1 / 2 ) + 1 ( 1 ) = 1 / 2 , 
<y 2 , y 2 > = ( — 1/2) 2 + (i/2) 2 + (I) 2 = 3/2, 



(x^yi) 

(yi-yi> 



(x 3 ,y 2 > 1 1/2 

ry 2 = x 3 - — yi - — — y 2 

(y 2 . y 2 > 2 3/2 
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The vectors yi , y 2 , and y 3 form an orthogonal set. To make this set orthonormal, we 
note that (yi, yi> = 2, (y 2 , y 2 ) = 3/2, and (y 3 , y 3 > = (2/3)(2/3) + (— 2/3)(— 2/3) + 
(2/3) (2/3) = 4/3. Therefore, 

llyill = \/{ÿï 7 ÿî> = V2 ||y2ii = v / (y2-y2> = 

Ily 3 ll = v / (y 3 ^y 3 > = 2/V3, 


and 



Example 2 Use the Gram-Schmidt orthonormalization process to construct an 
orthonormal set of vectors from the linearly independent set {xi, X2, X3, X4}, where 



Solution 



(yi, yi) = i(i) + i(i) + 0(0) + i(i) = 3, 

(x 2 ,yi> = l(l) + 2(l) + l(0) + 0(l) = 3, 
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(y 2 , y 2 > = 0(0) + i(D + Kl) + (— 1)(— 1) = 3, 


(x 3 , yi) = 0(1) + 1(1) + 2(0) + 1(1) = 2, 
(x 3 , y 2 > = 0(0) + 1(1) + 2(1) + 1(-1) = 2, 



(X3,yi> 

ryi 

(yi-yi) 


(X3,y 2 > 

-, ry 2 

(y 2 ,y 2 > 



-2/3 

-1/3 

4/3 

1 


(y3,y3) = 



10 

T’ 


(x 4 . yi> = 1(1) + 0(1) + 1(0) + 1(1) = 2, 

<X 4 , y 2 > = KO) + 0(D + Kl) + 1(— i) = o, 

( x 4 , y 3 > = i(x) + °(y) + 1 ( 5 ) + 1(1) = b 


y 4 = x 4 


(x 4,yi) 

(yi.yi) 


yi 


x 4 - jJl 


0 

3 


y2 


<x 4 , y 2 > 

<x 4 , y3> 

(y 2 ,y 2 > 

(y3,y3> 


2/3" 

5/3 

-1/2 

10/3 y3 “ 

1/3 ' 


1/6_ 


y3 


Th en 


(y 4 . y 4 > = (2/3)(2/3) + ( — 1/2)( — 1/2) + (l/3)(l/3) + (— 1/6)(— 1/6) 
= 5/6, 


and 



"1 


"1/V3" 

1 

1 


l/y/3 

V3 

0 


0 


1 


l/y/3 



0 


■ 0 ■ 

1 

1 

| 


l/y/3 

V3 

1 


l/y/3 


-1 


_-l/y/3_ 


q 2 = 
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qs = 


-2/3 

-1/3 

4/3 

1 


-2/V3Ô" 
— 1/V3Ü 
4/V3Ô ’ 
3/V3Ô_ 


q 4 = 


J5J6 


2/3 

■ 1/2 

1/3 

■ 1/6 


4/V3Ô" 
-3/V3Ü 
2/V3Ô ' 
■1/V3Ü. 


Problems 10.2 


1. Détermine which of the following vectors are orthogonal: 


x = o . y = 


, Z = 


, U = 


-4 


2. Détermine which of the following vectors are orthogonal: 


"il rr 

x — 1 , y = 1 , z = 

2 J L 1 

3. Find x so that 


1 , u = 
1 


-1 , V 

0 


3 x 

^ is orthogonal to ^ 


4. Find x so that 


-il [1 

x is orthogonal to 2 
3 3 


5. Find x so that [x x 2] is orthogonal to [1 3 — 1]. 

6. Find x and y so that [x y] is orthogonal to [1 3]. 

7. Find x and y so that 


x 11 

_y is orthogonal to both 2 and 1 
1 3 1 


3" 

6 ' 
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8. Find x, y , and z so that 




"1 


"1 

J 

Z 

is orthogonal to both 

0 

1 

and 

1 

2 


9. Redo Problem 8 with the additional stipulation that [x y z] J be a unit 
vector. 


In Problems 10 through 18, use the Gram-Schmidt orthonormalization process 
to construct an orthonormal set from the given set of linearly independent vectors. 


10. xi = 


1' 

2 


x 2 = 


' 2 ' 

1 


11. xi = 


ï 

1 


x 2 = 


' 3 ' 

5 


12. xi = 


3' 

■2 


x 2 = 


'3' 

3 


13. xi = 


Y 

2 

1 


x 2 = 


t 

0 

1 


x 3 = 


T 

0 

2 


14. xi = 


' 2 ' 

1 

0 


x 2 = 


' 0 ' 

1 

1 


x 3 = 


' 2 ' 

0 

2 


15. xi = 


T 

1 

0 


x 2 = 


' 2 ' 

0 

1 


x 3 = 


' 2 ' 

2 

1 


16. xi = 


' 0 ' 

3 

4 


x 2 = 


'3' 

5 

0 


x 3 = 


' 2 ' 

5 

5 


17. xi = 


' 0 ' 

1 

1 

1 


x 2 = 


T 

0 

1 

1 


x 3 = 


T 

1 

0 

1 



~r 


0" 


1" 


r 


i 


1 


0 


0 

x i = 

0 

, x 2 = 

-1 

, x 3 = 

-1 

, x 4 = 

0 


0 


0 


0 


-i 


x 4 = 


T 

1 

1 

0 


19. Prove that no y-vector in the Gram-Schmidt orthonormalization process 
is zéro. 
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20. Prove that the y-vectors in the Gram-Schmidt orthonormalization process 
form an orthogonal set. (Hint: first show that (y 2 , yi) = 0, hence y 2 must be 
orthogonal to yi. Then use induction.) 

21. With q j defined by Eq. (5), show that Eq. (4) can be simplified to y j = xj — 

EÜ ( x 7' 

22. The vectors 



r r 


"0 


1 " 

XI = 

1 

, X 2 = 

1 

, X3 = 

0 


0 


1 


-1 


are linearly dépendent. Apply the Gram-Schmidt process to it, and use the 
results to deduce what occurs whenever the process is applied to a linearly 
dépendent set of vectors. 

23. Prove that if x and y are orthogonal, then 

ll x - yll 2 = ll x ll 2 + llyll 2 - 

24. Prove that if x and y are orthonormal, then 

|| s\ + ty\\ 2 — s 2 + î 2 

for any two scalars 5 and t. 

25. LetQ be any n x n matrix whose columns, when considered as n-dimensional 
vectors, form an orthonormal set. What can you say about the product Q T Q? 

26. Prove that if (y, x) = 0 for every n-dimensional vector y, then x = 0. 

27. Let x and y be any two vectors of the same dimension. Prove that x + y is 
orthogonal to x — y if and only if ||x|| = ||y||. 

28. Let A be an n x n real matrix and p be a real n-dimensional column vector. 
Show that if p is orthogonal to the columns of A, then (Ay, p) = 0 for any 
/î-dimensional real column vector y. 

10.3 Projections and QR-Decompositions 

As with other vector operations, the inner product has a geometrical interprétation 
in two or three dimensions. For simplicity, we consider two-dimensional vectors 
here; the extension to three dimensions is straightforward. 

Let u and v be two nonzero vectors, considered as directed line segments (see 
Section 1.7), positioned so that their initial points coincide. The angle between 
u and v is the angle 0 between the two line segments satisfying 0 < 6 < n. See 
Figure 10.1. 

Définition 1 If u and v are two-dimensional vectors and 6 is the angle between 
them, then the dot product of these two vectors is u • v = | |u| | | |v| | cos 0 . 
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Figure 10.1 


To use Définition 1, we need the cosine of the angle between two vectors, which 
requires us to measure the angle. We shall take another approach. 

The vectors u and v along with their différence u — v form a triangle (see 
Figure 10.2) having sides ||u||, ||v||, and ||u — v||. It follows from the law of cosines 
that 

||u - v|| 2 = I |u| I 2 + ||v|| 2 - 2||u|| ||v|| cos 0 , 

whereupon 

u — v|| 2 ] 

(u - v, u - v)] 

= (u, v). 


|U||||V||COS0= 2 [llu|| -+- llvir — 


= - [(u, u) + (▼,▼)- 



Figure 10.2 
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Thus, the dot product of two-dimensional vectors is the inner product of those 
vectors. That is, 


U • Y = U 


v|| cos 6 = (u, y). 


(6) 


The dot product of nonzero vectors is zéro if and only if cos0 = 0, or 0 = 90°. 
Consequently, the dot product of two nonzero vectors is zéro if and only if the 
vectors are perpendicular. This, with Eq. (6), establishes the équivalence between 
orthogonality and perpendicularity for two-dimensional vectors. In addition, we 
may rewrite Eq. (6) as 


cos 6 = 


(U, V) 


u 


(7) 


and use Eq. (7) to calculate the angle between two vectors. 


Example 1 Find the angle between the vectors 


u = 



and v = 


-3' 

4 


Solution (u, v) = 2(— 3) + 5(4) = 14, ||u|| = V4 + 25 = \/29, ||v|| = V9 + 16 = 
5, so cos 6 = 14/(5\^9) = 0.1599, and 0 — 58.7°. ■ 

Eq. (7) is used to define the angle between any two vectors of the same, but 
arbitrary dimension, even though the geometrical significance of an angle becomes 
meaningless for dimensions greater than three. (See Problems 9 and 10.) 

A problem that occurs often in the applied sciences and that has important 
ramifications for us in matrices involves a given nonzero vector x and a nonzero 
reference vector a. The problem is to décomposé x into the sum of two vectors, 
u + v, where u is parallel to a and v is perpendicular to a. This situation is illustrated 
in Figure 10.3. In physics, u is called the parallel component of x and v is called the 



Figure 10.3 
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perpendicular component of x, where parallel and perpendicular are understood 
to be with respect to the reference vector a. 

If u is to be parallel to a, it must be a scalar multiple of a, in particular u = Àa. 
Since we want x = u + v, it follows that v = x — u = x — Àa. Finally, if u and y are 
to be perpendicular, we require that 

0 = (u, y) = (Àa, x — Àa) 

= À(a, x) — À 2 (a, a) 

= A[(a,x) - A. (a, a)]. 

Thus, either À = 0 or X = (a, x)/(a, a). If X = 0, then u = Àa = 0, and x = 
u + v = v, which means that x and a are perpendicular. In such a case, (a, x) = 0. 
Thus, we may always infer that X = (a, x)/(a, a), with 

(a, x) (a, x) 

u = a and v = x a. 

(a, a) (a, a) 

In this context, u is the projection of x onto a, and v is the orthogonal complément. 


Example 2 Décomposé the vector 



into the sum of two vectors, one of which is parallel to 



and one of which is perpendicular to a. 


Solution 



Then, x = u + v, with u parallel to a and v perpendicular to a. ■ 


We now extend the relationships developed in two dimensions to vectors in 
higher dimensions. Given a nonzero vector x and another nonzero reference 
vector a, we define the projection of x onto a as 


pr°j a x = 


(a, x) 

a 

(a, a} 


( 8 ) 
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As a resuit, we obtain the very important relationship that 


x — — — - a is orthogonal to a. 

(a, a) 



That is, if we sub tract from a nonzero vector x its projection onto another nonzero 
vector a, we are left with a vector that is orthogonal to a. (See Problem 23.) 

In this context, the Gram-Schmidt process, described in Section 10.2, is almost 
obvious. Consider Eq. (4) from that section: 



(4 repeated) 


The quantity inside the summation sign is the projection of x-, onto y*. Thus for 
each k (k = 1, 2, . . . , j — 1), we are sequentially subtracting from xj its projection 
onto y*, leaving a vector that is orthogonal to y*. 

We now propose to alter slightly the steps of the Gram-Schmidt orthonormal- 
ization process. First, we shall normalize the orthogonal vectors as soon as they 
are obtained, rather than waiting until the end. This will make for messier hand 
calculations, but for a more efficient computer algorithm. Observe that if the y^ 
vectors in Eq. (4) are unit vectors, then the denominator is unity, and need not be 
calculated. 

Once we hâve fully determined a y^ vector, we shall immediately subtract the 
various projections onto this vector from ail succeeding x vectors. In particular, 
once yi is determined, we shall sub tract the projection of X2 onto yi from X2, then 
we shall subtract the projection of X3 onto yi from X3, and continue until we hâve 
subtracted the projection of x n onto yi from x n . Only then will we return to X2 and 
normalize itto obtain y2- Then, we shall subtract from X3, X4, . . . , x n the projections 
onto y 2 from X3, X4, . . . , x w , respectively, before returning to X3 and normalizing 
it, thus obtaining y3. As a resuit, once we hâve yi, we alter X2, X3, . . . , x n so each 
is orthogonal to yi; once we hâve y2, we alter again X3, X4, . . . , x n so each is also 
orthogonal to y2; and so on. 

These changes are known as the revised Gram-Schmidt algorithm. Given a set 
of linearly independent vectors {xi, X2, . . . , x n ], the algorithm may be formalized 
as follows: Begin with k — 1 and, sequentially moving through k — n\ 

(i) calculate r kk = ^/{x k ,x k ), 

(ii) set q A = (1 /r kk )x k , 

(iii) for j = k + 1, k + 2, . . . , n, calculate r^j — (x 7 -, q*), 

(iv) for j = k + 1, k + 2, . . . , n, replace xj by xj — r^/q*. 

The first two steps normalize, the third and fourth steps sub tract projections from 
vectors, thereby generating orthogonality. 



Chapter 1 0 Real Inner Products and Least-Square 


Example 3 Use the revised Gram-Schmidt algorithm to construct an orthonor- 
mal set of vectors from the linearly independent set {xi, X2, X3}, where 



" 1 “ 


"0" 


" 1 “ 

XI = 

1 

0 

, X 2 = 

1 

1 

, X 3 = 

0 

1 


Solution 

First Itération (/: = !) 


ni = vAi’Xi) = x/2, 


qi 

n 2 

r 13 


l/y/2 
l/y/2 
0 

(x, ,„) = -)=. 




" 0 " 

1 

'1/y/T 


"—1/2" 

X2 ^ x 2 - ri 2 qi = 

1 

1 

V2 

l/y/2 

0 

— 

1/2 

1 


" 1 “ 

1 

'1/y/T 


1/2“ 

x 3 ^ x 3 - ri 3 qi = 

0 

1 

V2 

l/y/2 

0 

= 

-1/2 

1 


Note that both X2 and X3 are now orthogonal to qi. 


Second Itération ( k = 2) 

Using vectors from the first itération, we compute 


ni = V / (X 2 ,X 2 > = ^3/2, 


1 

q2 = — X 2 



“—1/2 


~-l/y/6~ 

1/2 

— 

1/V6 

1 


2/V6_ 


1 

r 23 = (X 3 , q 2 > = 



1/2" 

1 

~-l/y/6~ 


2/3“ 

X3 X 3 — T 2 3 q 2 = 

-1/2 

1 

_I_ 

V6 

l/y/6 

2/V6_ 

— 

-2/3 

2/3 
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Third Itération ( k — 3) 

Using vectors from the second itération, we compute 


7*33 


^(X 3 ,X 3 ) — 


2 

VT 


1 1 

2/3" 


1/V3" 

q 3 — x 3 — r- 

m 2/V3 

-2/3 

— 

-1/V3 

2/3 


1/V3_ 


The orthonormal set is {qi, q2, q3}. Compare with Example 1 of Section 10.2. ■ 

The revised Gram-Schmidt algorithm has two advantages over the Gram- 
Schmidt process developed in the previous section. First, it is less effected by 
roundoff errors, and second, the inverse process— recapturing the x-vectors from 
the q-vectors— becomes trivial. To understand this second advantage, let us redo 
Example 3 symbolically. In the first itération, we calculated 

1 

qi = — xi, 

ni 


so, we immediately hâve, 

xi = mqi- (10) 

We then replaced X2 and X3 with vectors that were orthogonal to qi. If we dénoté 
these replacement vectors as \' 2 and X3, respectively, we hâve 

*2 = x 2 - mqi and X3 = x 3 - ri 3 qi. 


With the second itération, we calculated 



1 

— (X2 - mqi). 


Solving for X2, we get 


x 2 = mm + mm- ( 11 ) 

We then replaced X3 with a vector that was orthogonal to q2. If we dénoté this 
replacement vector as X3 , we hâve 

x 3 = x 3 “ ^23 q 2 = (X 3 ri3qi) - r 23 q 2 . 

With the third itération, we calculated 

1 1 

q3 = — X3 = — (x 3 - ri 3 qi - r 2 3q2). 
r33 r33 
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Solving for X3, we obtain 


X 3 = ri 3 qi + ^3q2 + (12) 

Eqs. (10) through (12) form a pattern that is easily extended. Begin with 
linearly independent vectors xi, X2, . . . , x n , and use the revised Gram-Schmidt 
algorithm to form qi, q2, • • • , q«. Then, for any k(k = 1,2 , ,n). 

*k = n k qi + r 2 *q2 + r 3 *q3 H h r kk q k . 

IfwesetX=[xi X2 ... x w ], 


Q = [qi q2 q„] 

and 



ni 

n 2 

ri3 

... r\ n 


0 

ni 

^23 

• • • rin 

R = 

0 

0 

• 

G3 

• 

• • • rz n 
• 


0 

• 

0 

0 

• 

... V 

' nn 


we hâve the matrix représentation 

X = QR, 




which is known as the QR-decomposition of the matrix X. The columns of Q form 
an orthonormal set of column vectors, and R is upper (or right) triangular. 

In general, we are given a matrix X and are asked to generate its QR- 
decomposition. This is accomplished by applying the revised Gram-Schmidt 
algorithm to the columns of X, providing those columns are linearly independent. 
Then Eqs. (13) and (14) yield the desired factorization. 


Example 4 Construct a QR-decomposition for 



0 1 

1 0 . 

1 1 


Solution The columns of X are the vectors xi, X 2 , and X 3 of Example 3. Using 
the results of that problem, we generate 



1 /V 2 

- 1 /V 6 

1/V3 


"V2 

l/y/2 

l/y/2 

Q = 

1 /V 2 

1 /V 6 

-1/V3 

and R = 

0 

7372 

1/V6 


0 

2 /V 6 

l/y/3 


0 

0 

2/V3 
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Example 5 Construct a QR-decomposition for 


110 1 
1210 
0121 ' 
10 11 

Solution The columns of X are the vectors 



"1 


~r 


" 0 " 


"1“ 


1 


2 


1 


0 

Xi = 

0 

, X 2 = 

1 

, x 3 = 

2 

, x 4 = 

1 


1 


0 


1 


1 


We apply the revised Gram-Schmidt algorithm to these vectors. Carrying eight 
significant figures through ail computations but rounding to four décimais for 
présentation purposes, we get 


First Itération (k — 1) 


ru = y 7 (xi, xi) = \/3 = 1.7321, 



"1 


"0.5774' 

1 1 

1 


0.5774 

q l = Xl = nz 

ni V3 

0 

— 

0.0000 


1 


0.5774 


r \2 = (x 2 , qi) = 1.7321, 
ri 3 = (x 3 ,qi) = 1.1547, 
ri4 = (x 4 ,qi) = 1.1547, 


x 2 x 2 ri 2 qi = 


x 3 ^ x 3 - ri 3 qi = 


" 1 " 


"0.5774" 


0.0000“ 

2 

- 1.7321 

0.5774 


1.0000 

1 

0.0000 


1.0000 

0 _ 


0.5774_ 


- 1.0000 

" 0 " 


“0.5774" 


“-0.6667“ 

1 

- 1.1547 

0.5774 


0.3333 

2 

0.0000 


2.0000 

1 


0.5774 


0.3333 

" 1 " 


“0.5774“ 


0.3333“ 

0 

- 1.1547 

0.5774 


-0.6667 

1 

0.0000 


1.0000 

1 


0.5774 


0.3333 


x 4 ^ x 4 ri 4 qi = 
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Second Itération ( k = 2) 

Using vectors from the first itération, we compute 


m — \/ (X2, X2> = 1.7321, 



0.0000 


0 . 0000 " 

1 1 

1.0000 


0.5774 

q2 = X2 = . 

r 2 2 1.7321 

1.0000 

— 

0.5774 


- 1.0000 


-0.5774 


^23 = (x3,q2) = 1.1547, 
r24 = (x4, q2) = 0.0000, 


x 3 ^ x 3 r 23 q2 = 


x 4 ^ x 4 - r 24 q2 = 


-0.6667 

0.3333 

- 1.1547 

2.0000 

0.3333_ 


0.3333” 

1 — 

-0.6667 

1.0000 

- 0.0000 

0.3333 



0.0000 


-0.6667 

0.5774 


-0.3333 

0.5774 


1.3333 

0.5774 


1.0000 

0 . 0000 " 


0.3333" 

0.5774 


-0.6667 

0.5774 


1.0000 

0.5774 


0.3333 


Third Itération (k — 3) 

Using vectors from the second itération, we compute 


^33 = y(x 3 ,x 3 ) = 1.8257, 



0.6667" 


"—0.3651" 

1 1 

—0.3333 


—0.1826 

q — 

r 33 " 1-8257 

1.3333 


0.7303 


1.0000 


0.5477 


r 34 = (x4, q 3 ) = 0.9129, 


X4 


x 4 - t 3 4 q3 = 


0.3333 


"—0.3651" 


0.6667“ 

-0.6667 

- 0.9129 

-0.1826 


-0.5000 

1.0000 

0.7303 


0.3333 

0.3333 


0.5477 


-0.1667 


Fourth Itération (k — 4) 


Using vectors from the third itération, we compute 

r 44 = y ( x 4i X 4 ) = 0.9129, 
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“ 0.6667“ 


“ 0.7303“ 

1 ^ 1 

-0.5000 


-0.5477 

r 44 X4 ~~ 0.9129 

0.3333 

— 

0.3651 


-0.1667 


-0.1826 


With these entries calculated (compare with Example 2 of Section 10.2), 
we form 


"0.5774 

0.5774 

0.0000 

0.5774 


0.0000 -0.3651 0.7303" 

0.5774 -0.1826 -0.5477 
0.5774 0.7303 0.3651 

-0.5774 0.5477 -0.1826 


and 



"1.7321 

0 

0 

0 


1.7321 

1.7321 

0 

0 


1.1547 

1.1547 

1.8257 

0 


1.1547' 

0.0000 

0.9129 

0.9129 


Finally, we note that in contrast to LU-decompositions, QR-decompositions 
are applicable to nonsquare matrices as well. In particular, if we consider a matrix 
containing just the first two columns of the matrix X in Example 5, and calculate 
ru, ri 2 , ^ 22 , qi, and q 2 as we did there, we hâve the décomposition 


"1 

1" 


"0.5774 

0.0000“ 

1 

2 


0.5774 

0.5774 

0 

1 


0.0000 

0.5774 

_1 

0_ 


_0.5774 

— 0.5774_ 


"1.7321 

0 


1.7321" 

1.7321 • 


Problems 10.3 


In Problems 1 through 10, détermine the (a) the angle between the given vectors, 
(b) the projection of xi onto X 2 , and (c) its orthogonal component. 
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5. xi = 





i 

(N 
1 


r7 
i 

6. xi = 

1 

, X 2 = 

0 


O 
1 


2 



"1 


c\T 

i 

7. xi = 

1 

, X 2 = 

2 


i 

o 

i 


1 



i 

0 

1 


C\1~ 
1 

8. xi = 

3 

4 

, X 2 = 

5 

5 


9. xi = 





X 2 = 


1' 

-2 

0 

-1 


In Problems 11 through 21, détermine QR-decompositions for the given matrices. 


11 . 


14. 


'1 2 ' 
2 1 

"1 2 ' 
2 2 
2 1 


12 . 


15. 


'1 3' 
1 5 

"1 1 ' 
1 0 
3 5 


13. 


3 3' 
■2 3 


16. 


3 

■2 

1 

1 


1' 

1 

1 

1 



2 

0 

2 


1 

2 2 



0 

3 

2 

17. 

1 

1 

0 

• 

00 

• 

1 

0 2 

• 

19. 

3 

5 

5 


0 

1 

2 


0 

1 1 



4 

0 

5 


"0 

1 

r 


"1 

0 

r 





20. 

1 

0 

i 

21. 

1 

1 

0 





1 

1 

0 

0 

-1 - 

-i 

• 





1 

1 

i 


0 

0 

0 






22. Show that 


(x,a) 

(a, a) 


a 


= |x|||cos0|, 


where 0 is the angle between x and a. 

23. Prove directly that 

(a, x) 

x- 7 r a 

(a, a) 

is orthogonal to a. 

24. Discuss what is likely to occur in a QR-decomposition if the columns are not 
linearly independent, and ail calculations are rounded. 
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10.4 The QR-Algorithm 

The QR-algorithm is one of the more powerful numerical methods developed 
for computing eigenvalues of real matrices. In contrast to the power methods 
described in Section 6.6, which converge only to a single dominant real eigenvalue 
of a matrix, the QR-algorithm generally locates ail eigenvalues, both real and 
complex, regardless of multiplicity. 

Although a proof of the QR-algorithm is beyond the scope of this book, the 
algorithm itself is deceptively simple. As its name suggests, the algorithm is based 
on QR-decompositions. Not surprisingly then, the algorithm involves numerous 
arithmetic calculations, making it unattractive for hand computations but idéal 
for implémentation on a computer. 

Like many numerical methods, the QR-algorithm is itérative. We begin with 
a square real matrix Ao. To détermine its eigenvalues, we create a sequence of 
new matrices Ai, A 2 , . . . , A*_i, A*, . . . , having the property that each new matrix 
has the same eigenvalues as Ao, and that these eigenvalues become increasingly 
obvious as the sequence progresses. To calculate A* (k = 1, 2, 3, . . .) once A^_i is 
known, first construct a QR-decomposition of A*_i: 

Afc-i = Q*-iRjfc-i, 

and then reverse the order fo the product to define 

Ak = Rk-iQk-i- 

It canbeshown that each matrix in the sequence {A*-} (k = 1,2,3, . . .)hasidentical 
eigenvalues. For now, we just note that the sequence generally converges to one 
of the following two partitioned forms: 


S ! T 

1 

0 0 0 • • • 0 ! a 




U 


i v " 

0 

0 

0 • 

0 

! b c 

0 

0 

0 • 

•• 0 

! d e 




If matrix (15) occurs, then the element a is an eigenvalue, and the remaining 
eigenvalues are found by applying the QR-algorithm a new to the submatrix S. 
If, on the other hand, matrix (16) occurs, then two eigenvalues are determined by 
solving for the roots of the characteristic équation of the 2x2 matrix in the lower 
right partition, namely 


À 2 — (b + e)X + (, be — cd) — 0. 
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The remaining eigenvalues are found by applying the QR-algorithm anew to the 
submatrix U. 

Convergence of the algorithm is accelerated by performing a shift at each 
itération. If the orders of ail matrices are n x n, we dénoté the element in the 
(n, n)-position of the matrix A^_i as iü*_i,and construct a QR-decomposition for 
the shifted matrix A^_i — Wk- il. That is, 

A*_i - wk - il = Q*_iR*-i- (17) 

We define 

A* = R*-iQ*-i + Wk- il. (18) 


Example 1 Find the eigenvalues of 




Solution Using the QR-algorithm with shifting, carrying ail calculations to eight 
significant figures but rounding to four décimais for présentation, we compute 


A 0 - (-7)1 


7 1 0 

0 7 1 

18 -1 0 


"0.3624 

0.1695 

-0.9165" 


"19.3132 

-0.5696 

0 . 0000 " 

0.0000 

0.9833 

0.1818 


0.0000 

7.1187 

0.9833 

0.9320 

-0.0659 

0.3564 


0.0000 

0.0000 

0.1818 


= QoRo, 


Ai = RoQo + (-7)1 


"19.3132 

-0.5696 

0 . 0000 " 


"0.3624 

0.1695 

-0.9165" 

0.0000 

7.1187 

0.9833 


0.0000 

0.9833 

0.1818 

0.0000 

0.0000 

0.1818 


0.9320 

-0.0659 

0.3564 


+ 





' 0.0000 

0.9165 

0.1695 


2.7130 

-0.0648 

- 0.0120 


-17.8035 
1.6449 , 
-6.9352 
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Ai - (-6.9352)1 = 


"6.9352 
= 0.9165 
0.1695 

"0.9911 
= 0.1310 
0.0242 

= QiRi 


2.7130 

6.8704 

- 0.0120 

-0.1306 

0.9913 

-0.0153 


-17.8035" 

1.6449 

0.0000 

-0.02601 ("6.9975 3.5884 
0.0120 0.0000 6.4565 

0.9996 0.0000 0.0000 


= RiQi + (-6.9352)1 = 


-17.4294 

3.9562 

0.4829 


0.0478 2.9101 -17.5612 

0.9414 -0.5954 4.0322 

0.0117 -0.0074 -6.4525 


Continuing in this manner, we generate sequentially 


"0.5511 2.7835 -16.8072 

A 3 = 0.7826 -1.1455 6.5200 

0.0001 -0.0001 -6.4056 


and 


"0.9259 2.5510 -15.9729 

A 4 = 0.5497 -1.5207 8.3583 

0.0000 -0.0000 -6.4051 


A4 has form (15) with 


S = 


0.9259 


2.5510 

-1.5207 


and a — —6.4051. 


One eigenvalue is —6.4051, which is identical to the value obtained in Example 2 
of Section 6.6. In addition, the characteristic équation of R is À 2 + 0.5948À - 
2.8103 = 0, which admits both —2 and 1.4052 as roots. These are the other two 
eigenvalues of Aq. ■ 



Example 2 Find the eigenvalues of 



0 0 
0 0 
1 0 
0 1 
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Solution Using the QR-algorithm with shifting, carrying ail calculations to eight 
significant figures but rounding to four décimais for présentation, we compute 


A 0 - ( 6)1 = 


Ai = RoQo + (6)1 = 


Ai - ( 3 . 7810)1 = 


-6 

0 

0 -25 

1 

-6 

0 30 

0 

1 

-6 -18 

0 

0 

1 0 

- 0.9864 

- 0.1621 

0.1644 

- 0.9726 

0.0000 

0.1666 

0.0000 

0.0000 

" 6.0828 

- 0.9864 

0.0000 

6.0023 

0.0000 

0.0000 

0.0000 

0.0000 


QoRo- 


— 

- 0.1622 

- 0.0266 


0.9868 

- 0.0044 


0.0000 

0.9996 

_ 

0.0000 

0.0000 

i — 

- 3.9432 

- 0.0266 


0.9868 

- 3.7854 


0.0000 

0.9996 

_ 

0.0000 

0.0000 

i — 

- 0.9701 

- 0.2343 


0.2428 

- 0.9361 


0.0000 

0.2622 



0.0000 

0.0000 


" 4.0647 

- 0.8931 

X 

0.0000 

3.8120 

0.0000 

0.0000 


0.0000 

0.0000 

QiRi, 


i — 

- 0.3790 

- 1.6681 


0.9254 

0.9646 


0.0000 

0.5918 


0.0000 

0.0000 


- 0.0270 - 0.0046 
- 0.1620 - 0.0274 
- 0.9722 - 0.1643 
0.1667 - 0.9860 

0.0000 - 29.5918 
- 0.9996 - 28.1246 
6.0001 13.3142 

0.0000 2.2505 


4.9275 - 29.1787 
- 4.6881 27.7311 

2.3856 - 14.1140 ’ 
0.3751 3.7810 

4.9275 - 29 . 1787 “ 
- 4.6881 27.7311 

- 1.3954 - 14.1140 
0.3751 0.0000 

- 0.0628 - 0 . 0106 “ 
- 0.2509 - 0.0423 
- 0.9516 - 0.1604 
0.1662 - 0.9861 

- 5.9182 35 . 0379 “ 

2.8684 - 22.8257 
2.2569 8.3060 

0.0000 1.3998 


11.4235 - 33.6068 
- 7.4792 21.8871 

3.0137 - 8.5524 ' 

0.2326 2.4006 


A , = RjQi + ( 3 . 7810)1 = 
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Continuing in this manner, we generate, after 25 itérations, 


4.8641 

-4.4404 

18.1956 

-28.7675 

4.2635 

-2.8641 

13.3357 

-21.3371 

0.0000 

0.0000 

2.7641 

-4.1438 

0.0000 

0.0000 

0.3822 

1.2359 


which has form (16) with 



4.8641 

-4.4404" 

and 

b 

c 


"2.7641 

-4.1438" 

4.2635 

-2.8641 

_d 

e 


0.3822 

1.2359 


The characteristic équation of U is À 2 — 2k + 5 = 0, which has as its roots 1 ± 2i\ 
the characteristic équation oftheother 2 x 2 matrix is À 2 — 4A. + 4.9999 = 0, which 
has as its roots 2 ± i. These roots are the four eigenvalues of Aq. ■ 


Problems 10.4 


1. Use one itération of the QR-algorithm to calculate Ai when 



1 0 
0 1 . 
-1 7 


Note that this matrix differs from the one in Example 1 by a single sign. 

2. Use one itération of the QR-algorithm to calculate Ai when 



-17 7 

-4 1 . 

1 -14 


3. Use one itération of the QR-algorithm to calculate Ai when 



0 

0 

1 

0 


0 -13 
0 4 

0 -14 * 

1 4 


In Problems 4 through 14, use the QR-algorithm to calculate the eigenvalues 
of the given matrices: 

4. The matrix defined in Problem 1. 

5. The matrix defined in Problem 2. 
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"3 0 0 



"7 

2 0 



"3 2 

3" 


6. 

2 6 4 

• 

7. 

2 

1 6 

• 

8. 

2 6 

6 

• 


2 3 5_ 



0 

6 7 



_3 6 

11 



"20 

-1 " 


"1 

1 0 


3 

0 

5 " 

9. 

2 3 

2 

10. 

0 

1 1 

11 . 

1 

1 

1 


-1 0 

2 


5 

-9 ( 

< 

3 


-2 0 

3 


12. The matrix in Problem 3. 


"0 

3 

2 

-1" 


"10 

7 

8 

7" 

1 

0 

2 

-3 

14. 

7 

5 

6 

5 

3 

1 

0 

-1 

8 

6 

10 

9 

2 

-2 

1 

1 


7 

5 

9 

10 


1 0.5 Least-Squares 

Analyzing data for forecasting and predicting future events is common to business, 
engineering, and the sciences, both physical and social. If such data are plotted, 
as in Figure 10.4, they constitute a scatter diagram , which may provide insight 
into the underlying relationship between System variables. For example, the data 
in Figure 10.4 appears to follow a straight line relationship reasonably well. The 
problem then is to détermine the équation of the straight line that best fits the data. 
A straight line in the variables v and y having the équation 


y = mx + c, 




+» x 


Figure 10.4 
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y 



Figure 10.5 


where m and c are constants, will hâve one y-value on the line for each value of 
v. This y-value may or may not agréé with the data at the same value of x. Thus, 
for values of x at which data are available, we generally hâve two values of y, one 
value from the data and a second value from the straight line approximation to 
the data. This situation is illustrated in Figure 10.5. The error at each x, designated 
as e(x ), is the différence between the y-value of the data and the y-value obtained 
from the straight-line approximation. 


Example 1 Calculate the errors made in approximating the data given in 
Figure 10.5 by the line y = 2x + 1.5. 

Solution The line and the given data points are plotted in Figure 10.5. There 
are errors at x = 0, x = 1, x = 2, x = 3, and x = 4. Evaluating the équation 
y = 2x + 1.5 at these values of x, we compute Table 10.1. 

It now follows that 


e(0) = 1 - 1.5 = -0.5, 
Kl) = 5-3.5 = 1.5, 
e(2) = 3-5.5 = -2.5, 


e(3) = 6 — 7.5 = —1.5, 
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Table 10.1 



Evaluated from 

Given data 

y = 2x + 1.5 

x y 

y 

0 1 

1.5 

1 5 

3.5 

2 3 

5.5 

3 6 

7.5 

4 9 

9.5 


and 


e(4) = 9-9.5 = -0.5. 

Note that these errors could hâve been read directly from the graph. ■ 

We can extend this concept of error to the more general situation involving N data 
points. Let (x\, y\), (x 2 , yi), (* 3 , ys), • • • , C*7v, yyv) be a set of N data points for a 
particular situation. Any straight-line approximation to this data generates errors 
e(x\), e(x 2 ), e(x 3 ), . . . , e(xjv) which individually can be positive, négative, or zéro. 
The latter case occurs when the approximation agréés with the data at a particular 
point. We define the overall error as follows. 

Définition 1 The least-squares error E is the sum of the squares of the individual 
errors. That is, 


E = [e(xi)} 2 + [e(x 2 )f + [e(x 3 )f + ■■■ + [e(x N )f. 

The only way the total error E can be zéro is for each of the individual errors to be 
zéro. Since each term of E is squared, an equal number of positive and négative 
individual errors cannot sum to zéro. 


Example 2 Compute the least-squares error for the approximation used in 
Example 1. 

Solution 

E = [e(0)] 2 + [e(l)] 2 + [e(2 )] 2 + [e(3)] 2 + [e(4 )] 2 

= (— 0.5) 2 + (1.5) 2 + (— 2.5) 2 + (— 1.5) 2 + (-0.5) 2 
= 0.25 + 2.25 + 6.25 + 2.25 + 0.25 
= 11.25. ■ 
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Définition 2 The least-squares straight line is the line that minimizes the 
least-squares error. 


We seek values of m and c in (19) that minimize the least-squares error. For 
such a line, 

e(xi) = y; - (mxi + c), 

so we want the values for m and c that minimize 


N 

E — ^(y,- — mxi — c) 2 . 
i=l 


This occurs when 


dE 

dm 


and 


dE 

Ite 

or, upon simplifying, when 


N 

Y 2{yi — mxi — c)(—Xi) = 0 
i= 1 

N 

E 2 (yi - mx i - c )( _1 ) = 

i=l 




N 

+ nc= yi ■ 

i=l 



System (20) makes up the normal équations for a least-squares fit in two 
variables. 


Example 3 Find the least-squares straight line for the following v — y data: 


* 

0 

1 

2 

3 

4 

J 

1 

5 

3 

6 

9 


Solution Table 10.2 contains the required summations. 

For this data, the normal équations become 

30m + 10c = 65, 

10m + 5c = 24, 


which has as its solution m — 1.1 and c = 1.4. The least-squares straight line is 
y = 1.7* + 1.4. 
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Table 10.2 


Xi 

yt 

(Xi ) 2 

xtyi 

0 

1 

0 

0 

1 

5 

1 

5 

2 

3 

4 

6 

3 

6 

9 

18 

4 

9 

16 

36 

5 

Sum E x i — 10 

i=l 

X yt - 24 

i=l 

5 

E U /) 2 = 30 

1=1 

X Xjyt = 65 
i=l 


The normal équations hâve a simple matrix représentation. Ideally, we would 
like to choose m and c for (19) so that 


yi = + c 


for ail data pairs (jc/, y,*), / = 1, 2, . . . , N. That is, we want the constants m and c to 
solve the System 


mx i + c = yi, 
mx 2 + c = y 2 » 
mx 3 + c = y 3 , 


mijv + c = yw, 

or, equivalently, the matrix équation 


X\ 1 


yi 

X2 1 




y2 

X3 1 


m 



yi 

• • 

• • 


c 


• 

_X N 1 _ 


JN_ 


This System has the standard form Ax = b, where A is defined as a matrix 

having two columns, the first being the data vector [jq X 2 x 3 • • • ;qv] T , 

and the second containing ail ones, x = [m c] T , and b is the data vector 

[yi y >2 y 3 • • • y;v] T . In this context, Ax = b has a solution for x if and only if 
the data falls on a straight line. If not, then the matrix System is inconsistent, and 
we seek the least-squares solution. That is, we seek the vector x that minimizes 
the least-squares error as stipulated in Définition 2, having the matrix form 


E= ||Ax — b II 2 


( 21 ) 
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The solution is the vector x satisfying the normal équations, which take the matrix 
form 


A t Ax = A T b. (22) 

System (22) is identical to System (20) when A and b are defined as above. 

We now generalize to ail linear Systems of the form Ax = b. We are primarily 
interested in cases where the System is inconsistent (rendering the methods devel- 
oped in Chapter 2 useless), ands this generally occurs when A has more rows than 
columns. We shall place no restrictions on the number of columns in A, but we 
will assume that the columns are linearly independent. We seek the vector x that 
minimizes the least-squares error defined by Eq. (21). 

Theorem 1 Ifx has the property that Ax — b is orthogonal to the columns of A, 
then x minimizes ||Ax — b|| 2 . 

Proof. For any vector xo of appropriate dimension, 

|| Axo — b|| 2 = ||(Axo — Ax) + (Ax — b)|| 2 

= ((Axo — Ax) + (Ax — b) , (Axo — Ax) + (Ax — b)) 

= ((Ax 0 - Ax) , (Ax 0 - Ax)) + ((Ax - b) , (Ax - b)) 

= +2 ((Axo - Ax) , (Ax - b)) 

= ||(Ax 0 — Ax)|| 2 + || (Ax — b) || 2 
= +2 (Axo, (Ax — b)) — 2 (Ax, (Ax — b)) . 

It follows directly from Problem 28 of Section 10.2 that the last two inner products 
are both zéro (take p = Ax — b). Therefore, 

|| Ax 0 - b || 2 = || (Ax 0 - Ax)|| 2 + ||(Ax - b)|| 2 
> || (Ax - b) || 2 , 


and x minimizes Eq. (21). □ 

As a conséquence of Theorem 1, we seek a vector x having the property 
that Ax — b is orthogonal to the columns of A. Denoting the columns of A as 
Ai, A 2 , . . . , A„, respectively, we require 

(A/, Ax — b) = 0 (/ = 1, 2, . . . , n). 

if y = [vi y 2 • • • y n ] J dénotés an arbitrary vector of appropriate dimension, 
then 


Ay = Aiyi + A 2 y 2 H h A n y n , 
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and 


n 


(Ay, (Ax - b)) = ( ( Ax “ b ) 


i=l 


n 


= Y ^yi’ (Ax _ 


i = 1 
n 

Y yt (Ai, (Ax 

i = 1 
0 . 


-b)) 



It also follows from Problem 39 of Section 6.1 that 

(Ay, (Ax - b)> = (y, A^(Ax - b)> = (y, (A^Ax - A^b)). (24) 

Eqs. (23) and (24) imply that (y, (A T Ax — A T b)) = 0 for any y. We may deduce 
from Problem 26 of Section 10.2 that A T Ax — A T b = 0, or A T Ax = A T b, which 
has the same form as Eq. (22)! Therefore, a vector x is the least-squares solution 
to Ax = b if and only if it is the solution to A T Ax = A T b. This set of normal 
équations is guaranteed to hâve a unique solution whenever the columns of A are 
linearly independent, and it may be solved using any of the methods described in 
the previous chaptersl 


Example 4 Find the least-squares solution to 


x -h 2 y + z = 1, 
3x — y =2, 
2x + y - z — 2, 
x + 2y + 2z — 1. 


Solution This System takes the matrix form Ax = b, with 



2 

-1 

1 

2 




and 



1 

2 

2 

1 


Then, 



15 

3 1 


12 

A T A = 

3 

1 

10 5 
5 6 

and A T b = 

4 

1 
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and the normal équations become 


"15 

3 1 


x 


"12" 

3 

1 

10 5 
5 6 


J 

Z 

— 

4 

1 


Using Gaussian élimination, we obtain as the unique solution to this set of équa- 
tions x = 0.7597, y = 0.2607, and z = —0.1772, which is also the least-squares 
solution to the original System. ■ 


Example 5 Find the least-squares solution to 

Ox + 3 y = 80, 

2x + 5 y = 100, 

5x — 2 y = 60, 

—x -b 8 y = 130, 
lOx — y = 150. 

Solution This System takes the matrix form Ax = b, with 


Then, 



1 

3" 


" 80" 

2 

5 


_ _ 


100 

5 

-2 

, x = 

X 

, and b = 

60 

-1 

8 


_.y_ 


130 

10 

-1 


150 


A T A = 


" 131 
-15 


-15" 

103 


and 


A T b = 


"1950" 
1510 ’ 


and the normal équations become 


" 131 

-15' 


x 


'1950" 

-15 

103 


.J. 


.1510. 


The unique solution to this set of équations is x = 16.8450, and y = 17.1134, 
rounded to four décimais, which is also the least-squares solution to the original 
System. ■ 
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Problems 10.5 


In Problems 1 through 8, find the least-squares solution to the given Systems of 
équations: 


1. 2x + 3 y = 8, 

3x — y = 5, 
x + y = 6. 

3. x + 3 y = 65, 

2x — y — 0, 

3x + y = 50, 

2x + 2y = 55. 

5. 2x + 3 y — 4z = 1, 
x — 2y + 3z = 3, 
x + 4y + 2z = 6, 

2v + y — 3z = 1. 

7. x + y - z = 90, 
2x + y + z = 200, 
x + 2y + 2z = 320, 
3x — 2y — 4z — 10, 
3x + 2y — 3z = 220. 


2. 2x + y = 8, 
x -F y = 4, 
-x + y = 0, 
3x + y = 13. 

4 . 2x + y = 6, 
x + y = 8, 
—2x + y = 11, 
-x + y = 8, 
3x + y = 4. 


2x 

+ 

3? 

+ 

2z = 

~-25, 

2x 

— 

y 

+ 

3 z = 

= 30, 

3x 

+ 

4y 

— 

2 z = 

= 20, 

3x 

+ 

5y 

+ 

4z = 

= 55. 

x 

+ 

2 y 

+ 

2 z = 

= 1. 

2x 

+ 

3 y 

+ 

2 z = 

= 2, 

2x 

+ 

4y 

+ 

4z = 

- —2, 

3x 

+ 

5y 

+ 

4 z = 

= 1. 

x 

+ 

3 y 

+ 

2 z = 

= -1. 


9 . Which of the Systems, if any, given in Problems 1 through 8 represent a least- 
squares, straight line fit to data? 

10 . The monthly sales figures (in thousands of dollars) for a newly opened shoe 
store are: 


month 

1 

2 

3 

4 

5 

sales 

9 

16 

14 

15 

21 


(a) Plot a scatter diagram for this data. 

(b) Find the least-squares straight line that best fits this data. 

(c) Use this line to predict sales revenue for month 6. 

11. The number of new cars sold at a new car dealership over the first 8 weeks of 
the new season are: 


week 

1 

2 

3 

4 

5 

6 

7 

8 

sales 

51 

50 

45 

46 

43 

39 

35 

34 



10.5 Least-Squares 


(a) Plot a scatter diagram for this data. 

(b) Find the least-squares straight line that best fits this data. 

(c) Use this line to predict sales for weeks 9 and 10. 

12. Annual rainfall data (in inches) for a given town over the last seven years are: 


year 

1 

2 

3 

4 

5 

6 

7 

rainfall 

10.5 

10.8 

10.9 

11.7 

11.4 

11.8 

12.2 


(a) Find the least-squares straight line that best fits this data. 

(b) Use this line to predict next year’s rainfall. 


13. Solve System (20) algebraically and explain why the solution would be 
susceptible to round-off error. 

14. (Coding) To minimize the round-off error associated with solving the normal 
équations for a least-squares straight line fit, the (*/, y/) -data are coded before 
using them in calculations. Each xt -value is replaced by the différence between 
x, and the average of ail v/-data. That is, if 


N 


l 

X = — ^ Xi, then set x\ — x t — X, 


i=l 


and fit a straight line to the (v-, y/) -data instead. 

Explain why this coding scheme avoids the round-off errors associated 
with uncoded data. 

15. (a) Code the data given in Problem 10 using the procedure described in 
Problem 14. 


(b) Find the least-squares straight line fit for this coded data. 

16. (a) Code the data given in Problem 11 using the procedure described in 
Problem 14. 


(b) Find the least-squares straight line fit for this coded data. 

17. Census figures for the population (in millions of people) for a particular région 
of the country are as follows: 


year 

1950 

1960 

1970 

1980 

1990 

population 

25.3 

23.5 

20.6 

18.7 

17.8 


(a) Code this data using the procedure described in Problem 14, and then 
find the least-squares straight line that best fits it. 

(b) Use this line to predict the population in 2000. 
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18 . Show that if A = QR is a QR-decomposition of A, then the normal équations 
given by Eq. ( 22 ) can be written as R T Rx = R T Q T b, which reduces to 
Rx = Q T b. This is a numerically stable set of équations to solve, not subject 
to the same round-off errors associated with solving the normal équations 
directly. 

19. Use the procedure described in Problem 18 to solve Problem 1. 

20. Use the procedure described in Problem 18 to solve Problem 2. 

21. Use the procedure described in Problem 18 to solve Problem 5. 

22. Use the procedure described in Problem 18 to solve Problem 6. 

23. Détermine the error vector associated with the least-squares solution of 
Problem 1, and then calculate the inner product of this vector with each of the 
columns of the coefficient matrix associated with the given set of équations. 

24. Détermine the error vector associated with the least-squares solution of 
Problem 5, and then calculate the inner product of this vector with each of the 
columns of the coefficient matrix associated with the given set of équations. 




Appendix: A Word on 
Technology 


We hâve covered a number of topics which relied very heavily on computations. 
For example, in Chapter 6, we computed eigenvalues and in Chapter 9 we raised 
transitional matrices to certain powers. 

While it is true that much of the “number crunching” involved the basic oper- 
ations of addition and multiplication, ail would agréé that much time could be 
consumed with these tasks. 

We, as educators, are firm believers that students of mathematics, science, and 
engineering should first understand the underlying fundamental concepts involved 
with the topics presented in this text. However, once these ideas are mastered, 
a common sense approach would be appropriate regarding laborious numerical 
calculations. 

As the first decade of this new millennium is coming to a close, we can take 
advantage of many tools. Calculators and Computer Algebra Systems are idéal 
instruments which can be employed. 

We give a few suggestions below: 

• TI-89 calculator produced by Texas Instruments (http://www.ti.com/) 

• Various products developed by Maplesoft (http://www.maplesoft.com/) 

• Various Mathematica © software packages (http://www.wolfram.com/) 

Thank you for using our text. 
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Answers and Hints to Selected 
Problems 


CHAPTER 1 

Section 1.1 

1 . A is 4 x 5 , B is 3 x 3 , C is 3 x 4 , 

D is 4 x 4 , E is 2 x 3 , F is 5 x 1 , 

G is 4 x 2 , H is 2 x 2 , J is 1 x 3 . 

2. (213 = —2, ci2i — 2, 

b\3 = 3 , b2i = 0 , 

C 13 = 3 , C 21 = 5 , 

d\3 = t 2 , d2i = t - 2, 

e \3 = e 21 = §, 

/i3 = does not exist, /21 = 5 , 
g\3 — does not exist, g2i — 2n, 
h 13 — does not exist, /z2i = 0, 
y'i3 = — 30 , y'21 does not exist. 

3 . (223 = —6, *232 = 3 , 631 = 4 , 

6.32 = 3 , en = 1 , d — 22 1 4 , e \ 3 = g, 
#22 = 18 , g 23 and hs2 do not exist. 





'1 Z 3 " 
2 1 | 



4. A = 

1 - 1 " 
-1 1 

5. A = 

6. B = 

1 0 - 1 " 
0 -1 -2 




_ 3 1 1 . 


-1 -2 -3 


1111 
1 2 3 4 * 


8. D 


0 -1 -2 -3 

3 0 -1 -2 . 

4 5 0 -1 
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7. C = 


Answers and Hints to Selected Problems 


9. (a) [9 15], (b) [12 0], (c) [13 30], (d) [21 15]. 


10. (a) [7 4 1776], (b) [12 7 1941], 

(d) [10 31 1688], 


(c) [4 23 1809], 


11. [950 1253 98], 


"3 5 3 4 

12. 0 2 9 5 

4 2 0 0 


"72 12 16 
13. 45 32 16 

81 10 35 


"100 150 50 500" 

14. 27 45 116 2 . 

29 41 116 3 

[1000 2000 3000] 

15 * ( a ) [ o 07 0.075 0.0725_|’ W 


1070.00 2150.00 3217.50 


0.075 


0.08 0.0775 


0.95 0.05" 
0.01 0.99 ' 


"0.80 0.15 0.05" 
19. 0.10 0.88 0.02 . 

0.25 0.30 0.45 


0.6 0.4 
0.7 0.3 


"0.10 0.50 0.40 
18. 0.20 0.60 0.20 
0.25 0.65 0.10 


Section 1.2 




Answers and Hints to Selected Problems 


17. 




4 -3 



"-0.1 0.2" 

18. 

-1 4 

19. X = 

i — 
’xl- 

1 

0.9 -0.2 

• m • 

-10 6 

-8 0 


[4 4j 


20. Y = 


-11 - 12 ' 
-11 -19 


23. R = 


27. 


21. X = 


- e 3 + 6e 2 + e 6e -6 
2i - e 4 - 29 2 - e + 6/e 


" 11 

1 


1 

1 

O 

O 

Lh 

-3 

8 

22. Y = 

0.5 -1.0 

4 -3 


2.5 -1.5 

1 17 


1.5 -0.5 





L_ 

_i 



L_ 




"—1.5 

1.0" 




"-2.8 

-1.6' 

24. S = 

-1.0 

-1.0 

25. 

" 5 

8" 

3.6 

— 9.2_ 

-1.5 

1.0 

13 

9_ 




2.0 

0 





32. (a) [200 150], (b) [600 450], (c) [550 550], 

33. (b) [11 2 6 3], (c) [9 4 10 8], 

34. (b) [10,500 6,000 4,500], (c) [35,500 14,500 3,300], 


Section 1.3 


1. (a) 2x2, 
(f ) 2 x 4, 
(j) 1x4, 


(b) 4x4, (c) 2 x 1, (d) Not defined, (e) 4x2, 

(g) 4x2, (h) Not defined, (i) Not defined, 

(k) 4x4, (1) 4 x 2. 



22 
50 ‘ 



34 
46 ■ 



5-4 3 
9-8 7' 


5. A = 


13 -12 
17 -16 


11 
15 ‘ 


6. Not defined. 


7. [-5 -6], 


8. [-9 -10], 


9. [-7 4 -1], 


10. Not defined. 



1 -3 
7 -3 ‘ 



2-2 2 
7-41. 
-8 4 0 


13. [1 3], 


14. Not defined. 


15. Not defined. 


16. Not defined. 



"-1 

-2 

~r 


(N 
1 

-2 

1" 

17. 

1 

0 

-3 

• 

00 

• 

-2 

0 

0 


1 

3 

5 


t-H 
1 

-2 

— i 
(N 


19. [-1 1 5], 
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x + 2y 
3x + 4y 


_ 

x — z 


_ _ 

23. 

3x + y 4- z 
x 3y 

24. 

a\\x + ^ 12 ^ 
_Ü2\X + Ü22 y _ 



2b\\ - b 12 + 3/^13 
2/721 — /?22 + 3 /? 23 _ ’ 



0 0 
0 0 * 



0 40 
-16 8 • 



0 

0 

0 


0 0 

0 0 . 

0 0 



7 5 

11 10 * 




1 

1 

1 


1 


x 


"2 

3 


J 

— 

4 

0 




0 



3 2 4 


x 

1 0 1 


J 

2 2 0 


Z 

1 2 3 


w 



35. (a) PN = [38,000], which represents the total revenue for that flight. 


(b) NP = 


26,000 45,5000 

4.000 7,000 

2.000 3,500 


65.000 

10.000 , 

5,000 


which has no physical signifîcance. 


36. (a) HP = [9,625 9,762.50 9,887.50 10,100 9,887.50], which represents 

the portfolio value each day. 

(b) PH does not exist. 

37. TW = [14.00 65.625 66.50] T , which dénotés the cost of producing each 
product. 


38. OTW = [33,862.50], which dénotés the cost of producing ail items on order. 


39. FC = 


613 

887 

1870 


625 
960 , 
1915 


which represents the number of each sex in each State of sickness. 


Section 1.4 


1. 


"7 4 -1 


6 10 

2. 

2 2-6 



r + 3 t 
2 t 3 + t 2 
t 4 + t 2 + t 
t 5 


2t 2 + 3 3 

4 t 2 + t t 

2t 3 + r + 1 t -\- 1 
2r 4 0 


3. (a) BA t , (b)2A T + B, (c) (B T + C)A = B T A + CA, (d) AB + C T , 
(e) A t A t + A t A - AA t - AA. 


4. X T X = [29], and XX T 


4 6 8 

6 9 12 . 

8 12 16 
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1-2 3-4 

5. X T X= J , and XX T = [30]. 

ô —O y — 1Z 

-4 8 -12 16 

6. [2x 2 + 6xy + 4y 2 ] . 7. A, B, D, F, M, N, R, and T. 

8. E, F, H, K, L, M, N, R, and T. 9. Yes. 

10. No, see H and L in Problem 7. 11. Yes, see L in Problem 7. 

"-5 0 0" 

12. 0 9 0 . 14. No. 

0 0 2 

19. D 2 is a diagonal matrix with diagonal éléments 4, 9, and 25; D 3 is a diagonal 
matrix with diagonal éléments 8, 27, and —125. 

20. A diagonal matrix with diagonal éléments 1, 8, 27. 

23. A diagonal matrix with diagonal éléments 4, 0, 10. 25. 4. 

1 1 _!~| f o ^ _i" 

1 2 2 u 2 2 

28. A = B + C. 29. \ i 5 + -| o -2 • 

-2 5 -sJ L 1 2 0_ 

'6 | n 2 ' 

30. 3 0 -4 + \ 0 3 . 

1-4 2 J |_— 2 -3 0_ 


34. (a) P 2 


0.37 0.63" 
0.28 0.72 


and P 3 


0.289 0.711" 
0.316 0.684_|’ 


(b) 0.37, (c) 0.63, (d) 0.711, (e) 0.684. 

35. 1 — ^ 1 — > 1 — > 1, 1 — ^ 1 — > 2 — > 1, 1 — ^ 2 — ^ 1 — ^ 1, 1 — > 2 — > 2 — ^ 1. 

36. (a) 0.097, (b) 0.0194. 37. (a) 0.64, (b) 0.636. 

38. (a) 0.1, (b) 0.21. 39. (a) 0.6675, (b) 0.577075, (c) 0.267. 


"0 1 0 0 0 0 0 " 

1 0 0 0 0 1 0 

0 0 0 1 1 0 0 

40. M= 00 1 00 1 0. 
0 0 1 0 0 1 1 

0 10 110 0 

0 0 0 0 1 0 1 
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41. (a) M = 


0 2 0 1 
2 0 0 2 
0 0 0 1 
12 10 
0 10 1 


0 

1 

0 , 
1 
0 


(b) 3 paths consisting of 2 arcs connecting node 1 to node 5. 


42. (a) M = 


0 110 
10 10 
110 0 
0 0 0 0 
1111 
0 0 0 1 
0 0 0 1 
0 0 0 0 


10 0 0 
10 0 0 
10 0 0 
1110 
0 10 0 ’ 
10 0 1 
0 0 0 1 
0 110 


(b) M 3 has a path from node 1 to node 7; it is the first intégral power of M 
having mn positive. The minimum number of intermediate cities is two. 


Section 1.5 




" 4 

5 -1 

9 

1. (a), (b), and (d) are submatrices. 

3. 

15 
_ 1 

10 4 

1 5 

22 

9 


4. Partition A and B into four 2x2 submatrices each. Then, 


11 

9 

0 

0 

4 

6 

0 

0 

0 

0 

2 

1 

0 

0 

4 

-1 


18 

6 

0 

0" 


" 7 

8 

0 

0 

12 

6 

0 

0 

6. 

-4 

-1 

0 

0 

0 

0 

1 

0 

0 

0 

5 

1 

0 

0 

3 

4_ 


0 

0 

1 

2, 



"1 

0 

0 

0 

0 

0“ 


"1 

0 

0 

0 

0 

0 


0 

4 

0 

0 

0 

0 


0 

8 

0 

0 

0 

0 

a 2 = 

0 

0 

0 

0 

1 

0 

A 3 = 

0 

0 

0 

0 

0 

1 


0 

0 

0 

0 

0 

1 


0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


8. A" = A when n is odd. 
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Section 1.6 



3. [1 -0.4 1], 


4. (a) Not defined. 



12 3 
4 1 
24 6 ’ 
0 0 


(c) [29], (d) [29], 



"2 0-2 


1 0-1" 

5. (a) [4 -1 1], (b) [-1], (c) 

-1 0 1 
3 0-3 

, (d) 

0 0 0 

-1 0 1 


6. (c), (d), (f), (g), (h), and (i). 


7. (a) 72, 

8. (a) 72, 

9. (a)7Ï5, 


(b) 5, 

(c) 73, 

(d) ^73, 

(e) 715, 

(b) 75, 

(c) 73, 

(d) 2 , 

(e) V3Ô, 

(b) 739. 

12. x 

2~i r 3" 

4 5 “ 

" 10 " 
11 * 


(f) 739. 
(f) 72. 



3 


"4 


5“ 


6" 


"1 

13. v 

0 

+ y 

1 

+ z 

-2 

+ w 

8 

— 

0 


-1 


1 


2 


-1 


0 


16. [0.5 0.3 0.2], 


17. (a) There is a 0.6 probability that an individual chosen at random initially will 

live in the city; thus, 60% of the population initially lives in the city, while 
40% lives in the suburbs. 

(b) d (1) = [0.574 0.426]. (c) d (2) = [0.54956 0.45044], 

18. (a) 40% of customers now use brand X, 50% use brand Y, and 10% use other 

brands. 

(b) di = [0.395 0.530 0.075]. (c) d 2 = [0.38775 0.54815 0.06410], 

19. (a) d <0) = [0 1], (b) d (1) = [0.7 0.3]. 

20. (a) d <0) = [10 0], 

(b)d (2) = [0.21 0.61 0.18]. A probability of 0.18 that the harvest will begood 
in two years. 
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Section 1.7 
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CHAPTER 2 

Section 2.1 


1. (a) No. (b) Yes. 2. (a) Yes. (b) No. (c) Yes. 

3. No value of k will work. 4. k = 1. 5. k = 1/12. 

6. k is arbitrary; any value will work. 7. No value of k will work. 



3 

2 - 





1 1 2 
1 -1 -2 


1 2 2 





1 2 3 

1 -3 2 


3-4 7 





12. 50r + 60s = 70,000, 13. 5d + 0.25b = 200, 

30r + 40s = 45,000. 10 d+ b = 500. 

14. 8, 000 A + 3,0005+ 1,000C= 70,000, 

5, 000 A + 12,0005 + 10,000C = 181,000, 

1,000 A + 3,0005+ 2,000C = 41,000. 
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15. 5A + 4 B+ 8C + 12 D = 80, 
20A + 30 B + 15C + 5D = 200, 
3A + 3B+10C+ 1D= 50. 


16. b + 0.05c + 0.05.? = 20,000, 
c = 8,000, 

0.03c + ? = 12,000. 


17. (a) C = 800,000 + 30B, (b) Add the additional équation S = C. 


18. — 0 . 60/31 + 0 . 30/32 + 0 . 50/73 = 0 , 19. —\pi + \p2 + \p3 = 0 , 

0 . 40/71 - 0 . 75/32 + 0 . 35/73 = 0 , 

0.20/31 + 0.45/32 - 0.85/73 = 0. 4 pi 3 K + 3 P3 - U, 

\pi + 3 P2 - \ P3 = 0. 


20. — 0.85/71 + 0.10/?2 + 0.15/74 = 0, 

0.20/71 — 0.60/72 + 5/73 + 0. 4O/74 — 0, 

0.30/71 + 0.15/72 — 3/73 + 0.45/74 = 0, 

0.35/71 4- 0.35/72 + 5/73 — /74 = 0. 


22. A = 




and d 


20,000 

30,000 * 




0 

0.02 

0.50 




"50,000 


23. 

A = 

0.20 

0 

0.30 

and 

d = 

80,000 

• 



0.10 

0.35 

0.10 




30,000 




"0.20 

0.15 

0.40 

0.25" 




r 

0 1 

24. 

A 

0 

0.20 

0 

0 


and 

j 

5,000,000 

J\ — 

0.10 

0.05 

0 

0.10 


Cl = 


0 



0.30 

0.30 

0.10 

0.05 





0 


Section 2.2 


1. x = 1 , y = 1 , z = 2. 2. x — — 6z, y — lz,z is arbitrary. 

3. x = y = 1. 4. r = t + 13/7, s = 2f + 15/7, f is arbitrary. 

1 1 

5. / = -(—n + 1), m — ~(3n — 5/7 — 3), n and p are arbitrary. 

5 5 

6. x = 0, y = 0, z = 0. 7. x = 2, y = 1, z = — 1. 

8 . * = 1, y = 1, z = 0, w = 1. 


Section 2.3 


1. A b = 


"1 2 -3" 

2. A b = 

"1 2 -1 -1" 

3 1 1 


2-324 
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"12 5“ 


"2 4 0 2 

3. A b = 

-3 1 13 
4 3 0 

4. A b = 

3 2 18 

5 -3 7 15 


5. A b 


2 3 -4 12 

3-2 0 -1 . 

8 -1 -4 10 


6 . x + 2y = 5, 
7 = 8 . 


7. 

9. 

11 . 


12 . 

14. 

15. 

16. 

17. 

18. 

21 . 

23. 

25. 

26. 
27. 


x — 2y -\- 3z — 10, 
y - = -3, 

z — 4. 

x + 3 y = —8, 
j + 4z = 2, 
0= 0. 

u — v = —1, 
v — 2 w — 2, 

w — —3, 

0= 1. 


x = —11, y = 8. 


8. r — 3s + 12? = 40, 

^ — 6t = —200, 

t= 25. 

10 . a — 1b + 2c = 0 , 

— c = 0, 

0 = 0 . 


13. x = 32, _y = 17, z = 4. 


r — —410, ^ = —50, t — 25. 


x = — 14 + 12 z, _y = 2 — 4z, z is arbitrary. 
a — 5c, b — c, c is arbitrary. 


No solution. 


1 -2 5' 

0 1 23 


'12 3 4 

0 15 7 

0 0 1 41/29 


19. 


22 . 


1 6 5' 

0 1 18 

"13 2 

0 1 -2 


1 

4 


20 . 


0 0 1 -32/23 


1 3.5 -2.5' 
0 1 -6 


'1 3 
0 1 
0 0 
0 0 


2 

-5 

1 

0 


0 

1 

-9/35 

0 


24. 


'1 3/2 2 3 0 5 

0 1 -50 -32 -6 -130 

0 0 1 53/76 5/76 190/76 


x = 1, y = —2. 

x = 5/7 - (l/7)z, y = -6/7 + (4/7)z, z is arbitrary. 


« = —3, b — 4. 


28. r = 13/3, s = t = —5/3. 
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1 


29. 

r= 13 (21 + 80,J = 

13 ( 

30. 

x = l,y = î,z = 2. 

31. 

32. 

x = y = 1 . 

33. 


33. r = t + 13/7, s — 2t -\- 15/7, t is arbitrary. 


1 1 

34. / = -(—n + 1), m = -(3 n — 5p — 3),n and p are arbitrary. 
5 5 


35. r = 500, s = 750. 36. d = 30,b = 200. 37. A = 5, B = 8, C = 6. 

38. A = 19.759 - 4.145 £>, B = -7.108 + 2.735 Z), 

C = 1.205 — 0.277 D, D is arbitrary. 

39. b = $19,012. 

40. 80,000 barrels. 41. p\ = (48/33) p$, p 2 — (41/33)p$, p?, is arbitrary. 

42. pi = (8/9) pj, p 2 = (5/6)p3, p 3 is arbitrary. 

43. pi — 0.3435/?4, p 2 = 1.4195/74, 773 = 1.1489/74, /?4 is arbitrary. 

44. x\ = $66,000; X 2 = $52,000. 

45. To construct an elementary matrix that will interchange the ith and j th 
rows, simply interchange those rows in the identity matrix of appropriate 
order. 


46. To construct an elementary matrix that will multiply the ith row of a matrix by 
the scalar r, simply replace the unity element in the i-i position of an identity 
matrix of appropriate order by r. 


47. To construct an elementary matrix that will add r times the ith row to the 
/th row, simply do the identical process to an identity matrix of appropriate 
order. 


48. x (0) 


40.000 

60.000 ’ 



55,000 
43,333 ’ 



58.333 

48.333 



" 100 , 000 " 


" 83,200" 


"103,360" 

49. x (0) = 

160,000 

60,000 

x« = 

118,000 

102,000 

X® = 

127,240 

89,820 


The solution is x\ — $99,702; X 2 — $128,223; and X 3 = $94,276, roundedto the 
nearest dollar. 


50. x (0 > = 


0 


"1,500,000" 


" 2,300,00" 

10 , 000,000 

x (1 > = 

7,000,000 

X® = 

6,400,000 

0 

7 

500,000 

7 

800,000 

0 


3,000,000 


2,750,000 
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The solution is: energy = $2,484,488; tourism = $6,250,000; transportation = 
$845,677; and construction = $2,847,278, ail rounded to the nearest dollar. 


Section 2.4 


1. (a) 4, (b) 4, (c) 8. 

3. (a) 3, (b) 3, (c) 8. 

5. (a) 9, (b) 9, (c) 11. 

7. a = —3, b — 4. 


2. (a) 5, (b) 5, (c) 5. 

4. (a) 4, (b) -3, (c) 8. 

6. (a) 4, (b) 1, (c) 10. 

8. r = 13/3, s = t= -5/3. 


9. Depending on the roundoff procedure used, the last équation may not be 
0 = 0, but rather numbers very close to zéro. Then only one answer is 
obtained. 


Section 2.5 


1. Independent. 
4. Dépendent. 

7. Independent. 

10. Dépendent. 

13. Independent. 

16. Independent. 


2. Independent. 

5. Independent. 

8. Dépendent. 

11. Independent. 

14. Independent. 

17. Dépendent. 


3. Dépendent. 

6. Dépendent. 

9. Dépendent. 

12. Dépendent. 

15. Dépendent. 

18. Dépendent. 



"2 


"1 


1 


~r 

19. Dépendent. 20. 

1 

= (—2) 

1 

+ (1) 

0 

+ (3) 

i 


2 


0 


-1 


i 


21. (a) [2 3] = 2[1 0] + 3[0 1], (b) [2 3] = |[1 1] + (-/) [1 -1], (c) No. 




(b) No, 



"1 


"1 


"1 


1 " 

(c) 

1 

= (0) 

0 

+ (1) 

1 

+ (0) 

-1 


1 


1 


1 


1 


To 

1 


i 

H- * 

1 


1 

h-* 

1 


cT 

i 

0 

= d) 

0 

+ (1) 

0 

+ (0) 

0 

3 


1 


2 


1 


23 . 
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24. [a b] = (a)[l 0] + (Z?)[0 1], 

25. [a b] = [1 1] + (^) t 1 - 1 ]- 

26. [1 0] cannot be written as a linear combination of these vectors. 

27. [a -2a] = (a/2) [2 -4] + (0)[— 3 6]. 


28. [a b] = 


a + 2b\ ( 3a — b . 

'[1 3] + ( — - — ) [2 — 1] 4- (0)[1 1]. 


29. 


a 

b 

c 


7 


a — b + c 
2 


T 

0 

1 


+ 


7 


« + b — c 
2 


T 

1 

0 


+ 


—a + b + c 


0' 

1 

1 


30. No, impossible to write any vector with a nonzero second componeni 
linear combination of these vectors. 



a 


"1 


"1 


"2" 

31. 

0 

= (a) 

0 

+ (0) 

0 

+ (0) 

0 


a 


1 


2 


1 


32. 1 and 2 are bases. 


33. 7 and 11 are bases. 39. {—k)x + (l)^x = 0. 

42. 0 = AO = A(cixi + C 2 X 2 H + CkXk) — c\Ax\ + qAx 2 H- 1- c&Ax^ 

= ciyi + cïyi H 


Section 2.6 


1 . 2 . 2 . 2 . 


3. 1. 


4. 2. 


5. 3. 


6. Independent. 

9. Dépendent. 

12. Independent. 

15. Dépendent. 

18. Independent. 

21. Dépendent. 

23. (a) Yes, (b) Yes, (c) No. 
25. Yes. 26. Yes. 


7. Independent. 
10. Independent. 
13. Dépendent. 
16. Independent. 
19. Dépendent. 
22. Dépendent. 


8. Dépendent. 
11. Dépendent. 
14. Dépendent. 
17. Dépendent. 
20. Independent. 


24. (a) Yes, (b) No, (c) Yes. 
27. No. 28. First two. 


29. First two. 


30. First and third. 


31. 0. 


Section 2.7 

1. Consistent with no arbitrary unknowns; x = 2/3, y = 1/3. 

2. Inconsistent. 


as a 
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3. Consistent with one arbitrary unknown;x = (1/2) (3 — 2 z), y = —1/2. 

4. Consistent with two arbitrary unknowns; .r = (1/7) (11 — 5z — 2 w), 
y = (1/7)(1 - 3z + 3w). 

5. Consistent with no arbitrary unknowns; x = y = 1, z = — 1. 

6. Consistent with no arbitrary unknowns; * = y = 0. 

7. Consistent with no arbitrary unknowns; x = y = z = 0. 

8. Consistent with no arbitrary unknowns; * = y = z = 0. 

9. Consistent with two arbitrary unknowns; x = z — 7w, y = 2z — 2 w. 


CHAPTER 3 

Section 3.1 


1. (c). 

2. None. 


3. 

r 3 
14 

-5 
L 14 

—21 

14 

8 

14 J 

4. 

" 1 

3 

2 

L 3 

2“ 

3 

1 

3 J 

5. D has no inverse. 

7. 

4 

3 

-1" 

1 

8. 

" 3 

2 

—2 

1 " 

2 

1_ 

9. 

4 

-6 

-6" 

!2_ 



r-i 1 “| 

5 10 

3 -1 * 

_ 20 20 _ 


0 1 
1 0 


3 0 
0 1 







1 0 
0 -5 
0 0 


0" 

0 

15. 

~i o" 

3 1 

16. 

"l 3~ 
0 1 

17. 

"1 0 0" 
0 1 3 

1 


- 


- 


0 0 1 







"1 

0 

0 

0" 


"1 

0 

0" 


0 

1 

0 

8 

18. 

0 

1 

0 

19. 

0 

0 

1 

0 


5 

0 

1 







— 


— 


_0 

0 

0 

1_ 



"1 

0 

0 

1 

O 


"1 

0 

0 

0" 


0 

0 

0 

1 


0 

0 

0 

1 

21. 

0 

0 

1 

0 

. 22. 

0 

0 

1 

0 


_0 

1 

0 

0_ 


_0 

1 

0 

0_ 



10 0 0 
0 10 0 

0 0 10 

-2 0 0 1 

0 0 0 0 


0 

0 

0 . 
0 
1 


"1 0 
0 0 
0 0 
0 1 
0 0 
0 0 


0 0 
0 1 
1 0 
0 0 
0 0 
0 0 


0 0" 

0 0 
0 0 
0 0 * 
1 0 
0 1 
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24. 


1 0 
0 7 


25. 


"1 0 0 " 

26. 0 -0.2 0 . 
0 0 1 


2 0 
0 1 


28. 


1 - 2 " 

0 1 ' 


29. 


1 0" 

3 1 • 


30. 


1 0 

-1 1 


"1 0 0 ' 
31. 0 2 0 

_0 0 1 _ 

"1 0 -3 
34. 0 1 0 

0 0 1 


"0 1 0' 
32 . 10 0 

0 0 1 

"10 0 
35 . 0 12 

0 0 1 


33. 


"10 0 " 

0 10. 
-3 0 1 

"l 0 0 

36 . 0 1 0 

oo-4 


37. 


1 0 
0 1 
0 0 
0 0 


0 0 
0 0 
0 1 
1 0 


38. 


0 0 
1 0 
0 1 
0 0 


0 0 
0 0 
1 0 
0 1 


40. 


0 1 
0 0 
1 0 
0 0 


0 0 
0 0 
1 0 
0 1 


1 0 
0 1 
0 0 
0 0 


0 0 

0 0 

-2 0 

0 1 


43. 


2 0 
0 L* 


44. No inverse. 


0 -3 


47. 


TO 0 0 

0 1 0 

0 0 | 


"1 -1 0" 

48. 0 1 0 . 

0 0-1 


0 0 

-2 0 
0 f 


50 . 


0 0 

0 0 

1 0 

-2 1 



0 0 0 

2 0 0 
0 1 3 

0 0 1 



0 0 0 
5 0 0 

0 i 0 
0 0 1 


0 0 i 



i o o 
0 i 0 

0 0 1 
0 0 0 


MU) 
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Section 3.2 



4 

- 1 " 

„ i 

2 

- 1 " 

-3 

1 

2 * 3 

-1 

2 


3. Does not exist. 


4. 1 

4 

r 

5. 

2 -3“ 

6. 

4 -6 

11 

-3 

2 

• ^ t 

-5 8 

• 9 

-6 12 



1 1 - 1 " 


1 

O 

r-H 

0 

1 


"-i -i r 

1 -1 1 

8 . 

0 0 1 

9. 

6 5-4 

-1 1 1 


1 0 0 


-3 -2 2 


1 

10. Does not exist. 11. - 

"10 0" 
-5 2 0 

12- \ 

"3 -1 -8" 
0 2 1 

2 

1 -2 2 


0 0 3 



9 -5 -2 


1 7-2" 

13. 

5 -3 -1 

14 - 2 

7-2 3 


-36 21 8 

17 

-2 3 4 



17 


' 14 5 

-5 -3 
13 1 



16. Does not exist. 



5 

3 

1 

• 18 ‘ 7 

"0 

-4 

4" 

17 - ^ 

-6 

3 

12 

1 

5 

-4 

33 

-8 

15 

5 

4 

3 

7 

-8 



"4 • 

-4 

-4 

-4 


1 

0 

0 

0" 

0 

4 

2 

5 

. 20 . 

2 

-1 

0 

0 

0 

0 

2 

3 

-8 

3 

1 

1 

0 

0 

0 

0 

-2 


_— 25 

10 

Zs 

2 • 

-1_ 


21. Inverse of a nonsingular lower triangular matrix is lower triangular. 

22. Inverse of a nonsingular upper triangular matrix is upper triangular. 

23. 35 62 5 10 47 75 2 3 38 57 15 25 18 36. 

24. 14 116 10 20 -39 131 -3 5 -57 95 -5 45 36 72. 

25. 3 5 48 81 14 28 47 75 2 3 28 42 27 41 5 10. 

26. HI THERE. 27. THIS IS FUN. 

28. 24 13 27 19 28 9 0 1 1 24 10 24 18 0 18. 
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Section 3.3 

1. x = 1, y = —2. 2. a = —3, b — 4. 3. x — 2, _y = —1. 

4. / = 1, p = 3. 5. Not possible; A is singular. 

6 . \ = — 8 , _y = 5, z = 3. 7. x = y = z = 1. 

8. / = 1, m = —2, n = 0. 9. r = 4.333, s — t — —1.667. 

10. r = 3.767 , s = —1.133, £ = —1.033. 11. Not possible; A is singular. 

12. x = y = 1, z = 2. 13. r = 500, s = 750. 14. d = 30, b — 200. 

15. A = 5, B = 8 , C = 6 . 16. 5 = $19,012. 

17. 80,000 barrels. 18. x\ — 66,000; X 2 = 52,000. 

19. x\ = 99,702; jc 2 = 128,223; *3 = 94,276. 


Section 3.4 


11. A 2 = 


12. A -3 = 


11 -4' 
-8 3 


41 -15' 
-30 11 


B" 2 = 


B" 3 = 


9 -20' 
-4 9 


13. A- 2 = - 


1 

4 


22 - 10 ' 
-15 7 


B -4 = 


-38 85' 

17 -38 


1 


512 


47 15' 

-45 -13 


14. A -2 = 


15. A 3 = 


16. 


1 


125 


"1 -2 1 ' 
0 1 -2 
_0 0 1 

"1 -3 3' 

0 1 -3 

_0 0 1 

-11 - 2 ' 
2 -11 


B" 2 = 


B" 3 = 


'1 -4 4" 
0 1 2 
0 0 1 _ 

1 -6 -9 
0 1 3 

0 0 1 


17. First show that (B A 1 ) T = A 1 B T and that (A 1 B T ) 1 = (B t ) 1 A 


-1 


-1 


Section 3.5 


1 . 


1 0 
3 1 



"1 

r 


" 10" 


0 

i 

, X = 

_— 9_ 


' 1 0 
0.5 1 



"2 

1 " 


8 " 


0 

1.5 

, x = 

-5 


2 . 
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1 

0.625 


3 

0.125 ’ 


-400 
1275 * 


h - ' 

O 

O 
1 


"1 1 0 “ 


rn~ 
1 

1 1 0 


0 -1 1 

, x = 

1 

l ~o" 

1 

1 — > ‘ 

>— 1 ‘ 

1 


0 0 2 


(N 1 
1 

1 


1 

O 

0 

t-H 

1 


i 

0 

(N 

T— 1 

1 

1 


f ïn" 
i 

-1 1 0 


0 -1 1 

, X = 

2 

-2 -2 1 


1 

LT) 

O 

O 
1 


i 

1 

[ >— > - 


i 

o 

0 

t-H 

1 


"2 1 3“ 


“-10“ 

2 1 0 


0 -1 -6 

, x = 

0 

-1 0 1 


0 0 1 


10 


"1 0 0 


"3 2 1 



3 1 0 


0 -5 -1 

u 3 3 

, X = 

10 

-10 

i 

T— 1 

^l 00 

1 

t-H 

1 


o o 


40 



1 0 0 


"1 2 -1 


"79“ 

8. 

2 1 0 


0-4 3 

, x = 

1 


-1 -0.75 1 


0 0 4.25 


1 



"1 

0 

0" 


"1 

2 

-1“ 


" 19“ 

9 . 

0 

1 

0 


0 

2 

1 

, x = 

-3 


0 

0 

1 


0 

0 

1 


5_ 


"1 

0 

0“ 


"1 

0 

0" 


2“ 

10 . 

3 

1 

0 


0 

2 

0 

, x = 

-1 


1 

1 

2 

1 


0 

0 

2 


1 

2 



"1 

0 

0 

i 

o 


"1 

0 

1 

1 


1“ 

1 

1 

0 

0 


0 

1 

-1 

0 


-5 

1 

1 

1 

0 


0 

0 

1 

-1 

, x = 

2 

1 O 

1 

2 

1 


0 

0 

0 

3 


1 



10 0 0 


"2 1-1 3 



12. 

2 1 0 0 
0 0 10 


0 Z 5 _1 
u 2 2 2 

0 0-1 1 

, X = 

266.67 
-166.67 

166.67 


o 2 5 1 

u 7 7 1 


0 0 0 ^ 


266.67 
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"i o o oi ri 2 i il r io" 

1 1 0 0 0 -1 1 0 _ 10 

13 * 1 1 1 0 0 0 -1 1 ’ x_ 10 • 

0 -1 -2 lj [o 0 0 3 J [-10 

"1 0 0 0l[2 0 2 0] [-2.5" 

1 1 0002 -2 6 _ -1.5 

14 * -2 1.5 1 0 0 0 8 -8 ’ X_ 1.5 ‘ 

0.5 0 0.25 lj |_0 0 0 3J |_ 2.0 

15. (a) x = 5, _y = — 2; (b) * = — 5/7, y = 1/7. 

16. (a) x = 1, y = 0, z = 2i (b) x = 140, y — —50, z — —20. 



21. (d) A is singular. 


CHAPTER 4 

Section 4.1 
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Section 4.2 

Note : Assume ail variables are non-negative for (1) through (8). 

1. Let v = the number of trucks of wheat; y — the number of trucks of corn. 
2x -b 3y < 23, 3x + y < 17. 

The objective function is 5000x + 6000y. 

2. The objective function is 8000x + 5000y. 

3. Let v = the number of units of X; y = the number of units of Y. 2x + 3y > 180, 
3x + 2y > 240. The objective function is 500x + 750y. 

4. The objective function is 750x + 500y. 

5. Add the third constraint lOx + lOy > 210. 

6. Let x = the number of ounces of Zinc and y = the number of ounces of 
Calcium. 2x + y > 10, x + 4y > 15. The objective function is .04x + .05y. 

7. Add the third constraint 3x + 2y > 12. 

8. The objective function is .07x + .08y. 

9. The Richard Nardone Emporium needs at least 1800 cases of regular scotch and 
at least 750 cases of premium scotch. Each foreign shipment from distributor 
“x” can deliver two cases of the former and three cases of the latter, while 
distributor can produce nine cases of the former and one case of the latter 
for each foreign shipment. Minimize the cost if each “x” shipment costs $400 
and each shipment costs $1100. Note that the units for K (x, y) is in $100’s. 

(g) Three components are required to produce a spécial force (in pounds): 
mechanical, Chemical, and electrical. The following constraints are 
imposed: 

• Every x force requires one mechanical unit, two Chemical units and one 
electrical unit; 

• Every y force needs one mechanical unit, one Chemical unit and three 
electrical units; 

• Every z force requires two mechanical units, one Chemical unit and one 
electrical unit. 

The respective limits on these components is 12, 14, and 15 units, respectively. 
The Cafone Force Machine uses 2x plus 3y plus 4z pounds of force; maximize 
the sum of these forces. 

Section 4.3 

1. $50,000. 2. $57,000. 

3. $45,000. Note that the minimum occurs at every point on the line segment 
connecting (72,12) and (90,0). 
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4. $60,000. Note that the minimum occurs at every point on the line segment 
connecting (72,12) and (0,120). 

5. X = 72, Y = 12 is one solution, 6. About 29 cents. 

X = 90, Y = 0 is another solution. 


9. 400. 


12. 3280. 

14. 60,468.8. 


15. 3018.8. 

Section 4.4 



1. $50,000. 

2. $57,000. 


3. 30. 

4. 20. 


5. 72. 




XI X2 Si S 2 Z 



CHAPTER 5 

Section 5.1 


1. -2. 

2. 38. 

3. 38. 

(N 

1 

TT 

5. 82. 

(N 

00 

1 

• 

7. 9. 

8. -20. 

9. 21. 

10. 2. 

11. 20. 

12. 0. 

13. 0. 

14. 0. 

15. -93. 

16. 4 1 — 6. 

17. 2 1 2 + 6. 

18. 5 1 2 . 

19. 0 and 2. 

20. — 1 and 4. 


21. 2 and 3. 22. ±V6. 23. X 2 - 9X - 2. 

24. X 2 — 9X + 38. 25. X 2 - 13À - 2. 26. X 2 - 8X + 9. 

27. |A||B| = |AB|. 28. They differ by a sign. 

29. The new déterminants are the chosen constant times the old déterminants, 
respectively. 

30. No change. 31. Zéro. 32. Identical. 33. Zéro. 
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Section 5.2 


1 . 

—6. 

2. 

22. 

3. 

0. 

4. -9. 

5. 

-33 

6. 

15. 

7. 

-5. 

8. 

-10 

9. 0. 

10. 

0. 

11 . 

0. 

12. 

119. 

13. 

-8. 

14. 22. 

15. 

-7. 

16. 

-40 

17. 

52. 

18. 

25. 

19. 0. 

20. 

0. 

21. 

-11. 

22. 

0. 

23. 

Product of diagonal 

éléments. 

24. 

Always zéro. 


25. 

—À 3 

-|- 7 À -|- 22. 



26. 

—À 3 

+ 4A. 2 - 

17X. 

27. 

—À 3 

+ 6X — 9. 



28. 

—X 3 

+ 10À 2 - 

-22X- 

33. 






Section 5.3 

2. For an upper triangular matrix, expand by the first column at each step. 

3. Use the third column to simplify both the first and second columns. 

6. Factor the numbers —1,2,2, and 3 from the third row, second row, first column, 
and second column, respectively. 

7. Factor a five from the third row. Then use this new third row to simplify the 
second row and the new second row to simplify the first row. 

8. Interchange the second and third rows, and then transpose. 

9. Multiply the first row by 2, the second row by — 1, and the second column by 2. 

10. Apply the third elementary row operation with the third row to make the first 
two rows identical. 

11. Multiply the first column by 1 /2, the second column by 1 /3, to obtain identical 
columns. 

13. 1 = det(I) = det(AA _1 ) = det(A) det(A _1 ). 


Section 5.4 


i. -i. 

2. 0. 

3. -311. 

4. -10. 

5. 0. 

6. -5. 

7. 0. 

8. 0. 

9. 119. 

10. -9. 

11. -33. 

12. 15. 

13. 2187. 

14. 52. 

15. 25. 

16. 0. 

17. 0. 

18. 152. 

19. 0. 

20. 0. 


Section 5.5 


4 -1 
-3 1 • 


3. 


4 -6 
-6 12 


1. Does not exist. 2. 
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4 1“ 
-3 2 * 


2 -3" 
-5 8 * 


6. Does not exist. 




1 1 -1 

1 -1 1 

1 1 1 


"0 1 0 " 

8 . 0 0 1 . 

1 0 0 


-i -i r 

6 5 -4 . 

-3 -2 2 


10. Does not exist. 


1 T 1 0 0 

11. i -5 2 0 

2 1-2 2 


14 5 -6 

-5 -3 7 

13 1 -8 


13. Does not exist. 


ll~ 5 3 1_ l i[0-4 4 

14. — — — 6 3 12 . 15. j 1 5-4 

33 -8 15 5 4 3 7 -8 


a d — bc\ —c 


d —b 


17. det(AB) = det(A) det(B), 


19. Equals the number of rows in the matrix. 


Section 5.6 


1. x = 1, y = —2. 
4. s = 50, t — 30. 


2. x = 3, y = —3. 


3. a= 10/11, b= -20/11, 


5. Déterminant of coefficient matrix is zéro. 


6. System is not square. 7. x = 10, y = z = 5. 

8. x = 1, y = —4, z = 5. 9. x = y = 1, z = 2. 10. a = b = c = 1. 


11. Déterminant of coefficient matrix is zéro. 


13. x = 1, y = 2, z = 5, w = —3. 


12. r = 3, s = —2, t — 3. 


CHAPTER 6 


Section 6.1 


1. (a), (d), (e), (f ), and (h). 2. (a) 3, (d) 5, (e) 3, (f) 3, (h) 5. 


3. (c), (e), (f ), and (g). 


4. (c) 0, (e) 0, (f) -4, (g) -4. 


5. (b), (c), (d), (e), and (g). 6. (b) 2, (c) 1, (d) 1, (e) 3, (g) 3. 


7. (a), (b), and (d). 


8. (a) —2, (b) —1, (d) 2. 
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Section 6.2 


1. 2, 3. 

2. 1,4. 

• 

O 

00 

• 

4. -3,12. 

5. 3, 3. 

6. 3, —3. 

7. ±734- 

8. 3=4/. 

9. d=/. 

10. 1,1. 

H* 

• 

O 

O 

• 

h* 

Ni 

• 

O 

O 

• 

13. ±72. 

• 

h-* 

0 

1 

H-> ‘ 

h - 1 
• 

15. -10,11. 

16. t, —2t. 

17. 2 1, 2t. 

18. 20, 30. 

19. 2, 4, -2. 

20. 1,2,3. 

21. 1, 1,3. 

22. 0,2,2. 

23. 2, 3, 9. 

24. 1,-2, 5. 

25. 2, 3, 6. 

26. 0, 0, 14. 

27. 0, 10, 14. 

28. 2, 2, 5. 

29. 0,0,6. 

30. 3, 3, 9. 

31. 3, d=2/. 

32. 0, ±i. 

33. 3, 3, 3. 

34. 2,4, 1,±/V5. 

35. 1, 1,2,2. 



Section 6.3 
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23. 


" 10" 


T 


"2 


2" 

-6 


0 


0 


0 

11 

5 

0 

1 

1 

5 

1 

4 


0 


0 


-1 


24. 


'2/ y/5 

1/V5J 


1/V2 
’U/V2 


25. 


l/y/2' 



'l/y/5' 


3/>/Ï3" 


’l/\/5" 

1 

2/y/5_ 


_-2/VÎ3_ 

» 

_2/V5_ 


27. 


'0‘ 

1 

0 



"1/V3" 


~—l/y/2~ 


1/V3 

5 

0 


_1/V3_ 


l/y/2 


28. 


1/VÏ8 
■4/VÏ8 
1/VÎ8 J 


" 0 " 


■-1/V2' 

1 

9 

0 

0 


l/y/2 


29. 


0 

■l/y/2 



3/VÎ3" 


" 0 “ 

t 

-2/V13 

? 

4/5 


0 


3/5 


31. [-2 1], [1 1], 

33. [-3 2], [1 1], 

35. [1 0 1], [2 1 2], [-1 0 1], 
37. [1 -1 0], [1 1 1], [1 1 -2], 


30. [1 - 1], [-1 2]. 

32. [-2 1], [2 3], 

34. [1 -2 1], [1 0 1], [-1 0 1], 
36. [-2 -3 4], [1 0 0], [2 3 3]. 


38. Ax = A.x, so (Ax) t = (Àx) T , and x T A = Àx T . 39. [^ j j. 

«• [s §]• «• [s 1 !]• «• w [J !]• < b > s- 

43. [7/11 4/11]; probability of having a Republican is 7/11 = 0.636. 

44. [23/120 71/120 26/120]; probability of a good harvest is 26/120 = 0.217. 

45. [40/111 65/111 6/111]; probability of a person using brand Y is 
65/111 = 0.586 


Section 6.4 

1. 9. 2. 9.2426. 3. 5 + 8 + À = -4, À = -17. 

4. (5)(8)À = —4, À = —0.1. 5. Their product is -24. 

6. (a) -6, 8; (b) -15, 20; (c) -6, 1; (d) 1, 8. 

7. (a) 4, 4, 16; (b) -8, 8, 64; (c) 6, -6, -12; (d) 1, 5, 7. 

8. (a) 2A, (b)5A, (c)A 2 , 

9. (a) 2A, (b) A 2 , (c)A 3 , 


(d) A + 31. 
(d) A -21. 
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Section 6.5 



Section 6.6 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

0.6000 

1.0000 

5.0000 

2 

0.5238 

1.0000 

4.2000 

3 

0.5059 

1.0000 

4.0476 

4 

0.5015 

1.0000 

4.0118 

5 

0.5004 

1.0000 

4.0029 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

0.5000 

1.0000 

10.0000 

2 

0.5000 

1.0000 

8.0000 

3 

0.5000 

1.0000 

8.0000 
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Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

0.6000 

1.0000 

15.0000 

2 

0.6842 

1.0000 

11.4000 

3 

0.6623 

1.0000 

12.1579 

4 

0.6678 

1.0000 

11.9610 

5 

0.6664 

1.0000 

12.0098 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

0.5000 

1.0000 

2.0000 

2 

0.2500 

1.0000 

4.0000 

3 

0.2000 

1.0000 

5.0000 

4 

0.1923 

1.0000 

5.2000 

5 

0.1912 

1.0000 

5.2308 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

1.0000 

0.6000 

10.0000 

2 

1.0000 

0.5217 

9.2000 

3 

1.0000 

0.5048 

9.0435 

4 

1.0000 

0.5011 

9.0096 

5 

1.0000 

0.5002 

9.0021 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

1.0000 

0.4545 

11.0000 

2 

1.0000 

0.4175 

9.3636 

3 

1.0000 

0.4145 

9.2524 

4 

1.0000 

0.4142 

9.2434 

5 

1.0000 

0.4142 

9.2427 


7 . Itération Eigenvector components Eigenvalue 


0 

1.0000 

1.0000 

1.0000 


1 

0.2500 

1.0000 

0.8333 

12.0000 

2 

0.0763 

1.0000 

0.7797 

9.8333 

3 

0.0247 

1.0000 

0.7605 

9.2712 

4 

0.0081 

1.0000 

0.7537 

9.0914 

5 

0.0027 

1.0000 

0.7513 

9.0310 
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8. Itération Eigenvector components Eigenvalue 


0 

1.0000 

1.0000 

1.0000 


1 

0.6923 

0.6923 

1.0000 

13.0000 

2 

0.5586 

0.7241 

1.0000 

11.1538 

3 

0.4723 

0.6912 

1.0000 

11.3448 

4 

0.4206 

0.6850 

1.0000 

11.1471 

5 

0.3883 

0.6774 

1.0000 

11.1101 


9. Itération Eigenvector components Eigenvalue 


0 

1.0000 

1.0000 

1.0000 


1 

0.4000 

0.7000 

1.0000 

20.0000 

2 

0.3415 

0.6707 

1.0000 

16.4000 

3 

0.3343 

0.6672 

1.0000 

16.0488 

4 

0.3335 

0.6667 

1.0000 

16.0061 

5 

0.3333 

0.6667 

1.0000 

16.0008 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

0.4000 

1.0000 

0.3000 

-20.0000 

2 

1.0000 

0.7447 

0.0284 

-14.1000 

3 

0.5244 

1.0000 

-0.3683 

-19.9504 

4 

1.0000 

0.7168 

-0.5303 

-18.5293 

5 

0.6814 

1.0000 

-0.7423 

-20.3976 


11 . 

"1 

1 

is a linear combination of 

1 

-4 

and 

" 0 " 

1 


1 


1 


0 


which are eigenvectors 


corresponding to X = 1 and X = 2, not X = 3. Thus, the power method con- 
verges to À = 2. 

12. There is no single dominant eigenvalue. Here, \X\\ = |À 2 | = V34. 


13. Shift by À = 4. Power method on A = 
itérations to /x = —3. À + /z = 1. 

14. Shift by X = 16. Power method on A = 


-2 1 
2 -1 


converges after three 


13 2 3' 

2 -10 6 

3 6-5 


converges after 


three itérations to fi — — 14. À + /r , = 2. 
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Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

-0.3333 

1.0000 

0.6000 

2 

1.0000 

-0.7778 

0.6000 

3 

-0.9535 

1.0000 

0.9556 

4 

1.0000 

0.9904 

0.9721 

5 

-0.9981 

1.0000 

0.9981 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

-0.5000 


1 

-0.8571 

1.0000 

0.2917 

2 

1.0000 

-0.9615 

0.3095 

3 

-0.9903 

1.0000 

0.3301 

4 

1.0000 

0.9976 

0.3317 

5 

-0.9994 

1.0000 

0.3331 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

0.2000 

1.0000 

0.2778 

2 

-0.1892 

1.0000 

0.4111 

3 

-0.2997 

1.0000 

0.4760 

4 

-0.3258 

1.0000 

0.4944 

5 

-0.3316 

1.0000 

0.4987 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 


1 

-0.2000 

1.0000 

0.7143 

2 

-0.3953 

1.0000 

1.2286 

3 

-0.4127 

1.0000 

1.3123 

4 

-0.4141 

1.0000 

1.3197 

5 

-0.4142 

1.0000 

1.3203 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

1.0000 

0.4000 

-0.2000 

0.3125 

2 

1.0000 

0.2703 

-0.4595 

0.4625 

3 

1.0000 

0.2526 

-0.4949 

0.4949 

4 

1.0000 

0.2503 

-0.4994 

0.4994 

5 

1.0000 

0.2500 

-0.4999 

0.4999 
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20. Itération Eigenvector components Eigenvalue 


0 

1.0000 

1.0000 

1.0000 


1 

0.3846 

1.0000 

0.9487 

-0.1043 

2 

0.5004 

0.7042 

1.0000 

-0.0969 

3 

0.3296 

0.7720 

1.0000 

-0.0916 

4 

0.3857 

0.6633 

1.0000 

-0.0940 

5 

0.3244 

0.7002 

1.0000 

-0.0907 


Itération 

Eigenvector components 

Eigenvalue 

0 

1.0000 

1.0000 

1.0000 


1 

-0.6667 

1.0000 

-0.6667 

-1.5000 

2 

-0.3636 

1.0000 

-0.3636 

1.8333 

3 

-0.2963 

1.0000 

-0.2963 

1.2273 

4 

-0.2712 

1.0000 

-0.2712 

1.0926 

5 

-0.2602 

1.0000 

-0.2602 

1.0424 


22. Cannot construct an LU décomposition. Shift as explained in Problem 13. 

23. Cannot solve Lxi = y uniquely for xi because one eigenvalue is zéro. Shift as 
explained in Problem 13. 

24. Yes, on occasion. 


25. Inverse power method applied to A = 
À + l//z = 10 + 6 = 16. 


-7 

2 

3 


2 

4 

6 


26. Inverse power method applied 


to A = 


/z = 1/3. À + 1 /ii = -25 + 3 = -22. 


3 

6 

1 


converges to /i = 1/6. 


27 -17 1 

-17 21 1 


7 


1 11 


converges to 


CHAPTER 7 


Section 

1. (a) 


7.1 

'0 

0 

0 




Pki A) = 


PkiM) 

0 

0 


0 8 -16 
0-8 16 ; 
0 0 0 

0 0 
Pki^ 2 ) 0 

0 Pk(M) 



" 57 

78" 


"234 

348" 

117 

174 

•> 

522 

756 


2 . 


4 . In general, AB ^ BA. 
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5. Yes. 


0 2 " 

3 0 • 


7. 0 . 

3/2 


8. 2—2 element tends to oo, so limit diverges. 9. a , /?, d, and /. 


10. /. 11. Ail excep t c. 


13 I e °" 


V o o" 

15. 0 e~ 2 0 . 

0 0 1 


sin(l) 0 
0 sin(2) 


16. sin(A) = 


sin(Ài) 0 
0 sin(À 2 ) 


sin(-l) 


sin(28) 


1 0 

0 e 28 


sin(À„) 


k A 2k 


19. cos A = 

k = o 


~ (— 1) A 


( 2 *)! 


, cos 


1 0 
0 2 


cos(l) 0 
0 cos(2) 


cos (2) 0 0 

0 cos(— 2) 0 . 

0 0 1 


Section 7.2 


1. A 1 = 


3/2 -1/2 ‘ 


2. Since œq = 0, the inverse does not exist. 


3. Since œq = 0, the inverse does not exist. 


[-1/3 -1/3 2/3" 

4. A" 1 -1/3 1/6 1/6 . 

1/2 1/4 -1/4 


5. A" 1 = 


0 0 

0 0 

-1 0 

0 1 


Section 7.3 


1 = +ao, 


-2 3 


-1 = - a \ + œq ; [-1 2 

0 = Œq, 0 1 

— 1 = — a \ + û'q; 0 — 1 


0 = ao 


3 -6 


1 0 
0 1 


-3 6 
-1 2 


0 -1 
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3 78 = 3a\ + ao, 

4 78 = 4 78 = 4a i + a 0 ; 


— 4 78 + 2 (3 78 ) 
2 (4 78 ) - 2 (3 78 ) 


~4 78 + 3 78 
2 (4 78 ) - 3 78 



— 4 41 + 2 (3 41 ) 
2(4 41 ) -2(3 41 ) 


— 4 41 + 3 41 
2(4 41 ) - 3 41 


9. 1 = c /2 + ai + ao, 

1 = ci2— «1 + ao, 

2 222 = 4a? + 2 ai + ao; 


-1 0 (—4 + 4 (2 222 )) /3” 

0 1 (—2 + 2 ( 2 222 )) / 3 . 

0 0 2 222 


10. 3 17 = 9a? + 3ai + ao, 

5 47 = 25a? + 5ai + ao, 
10 17 = 100 a? + 10 ai + ao. 



8 a 3 + 4^2 + 2a\ + «o 
— 8 a 3 + 4^2 — 2 a i + ao 
27a 3 4- 9oc2 + 3ai 4- ao 
64a3 4- 16a2 4- 4a i 4- ao 


12. 1 = a 3 -b a 2 + ai + ao, 

(— 2 ) 25 = — 8 a 3 4- 4a2 — 2a i 4- ao, 

3 25 = 27a3 4- 9a2 4- 3ai 4- ao, 

(_4 25 ) = — 64a3 4- 16a2 — 4a i + ao. 

13 . 1 = a 4 4- a 3 4- a 2 4- ai 4- ao, 

1 = a 4 — a 3 4- a 2 — ai + ao, 

256 = 16a4 4- 8 a 3 + 4a2 4- 2a i 4- ao, 

256 = 16a4 — 8 a 3 4- 4 a 2 — 2a i 4- ao, 

6,561 = 81a4 4- 27a3 4- 9 a 2 4- 3ai 4- ao. 


14 . 


5,837 = 

9a2 4- 3ai 4- ao, 



381,255 = 

25a2 4- 5ai + ao, 


10 8 — 3 

' (10 ) 5 + 5 = 

100 a 2 4- 10ai 4- ao- 

15 . 

165 

= 8 a 3 4- 

4a2 4- 2a i + ao, 


357 

= — 8 a 3 + 

4 a 2 - 2a i 4- ao, 


5,837 

= 27a3 4- 

9 a 2 4- 3a i 4- ao, 


62,469 

= 64a3 4- 16a2 4- 4a i 4- ao- 

16 . 

3 

= a 3 4- 

a 2 4- ai 4- ao, 


357 

= — 8 a 3 4- 

4 a 2 - 2 a i + ao, 


5,837 

= 27a 3 4- 

9 a 2 4- 3ai + ao, 


68,613 

= — 64a 3 4- 

16a2 — 4a i 4- ao. 


17. 15 = a 3 -b a 2 + ai + ao, 

960 = — 8 a 3 4- 4 a 2 — 2ai 4- ao, 
59,235 = 27a3 + 9 a 2 + 3ai 4- ao, 
1,048,160 = — 64a3 4- 16a2 — 4ai 4- ao. 
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15 = «4 + Oi 3 -h Oi 2 + Œ\ + ao, 

— 13 = Œ 4 — Œ 3 + «2 — û'i + û'o, 

1,088 = 16ûf4 + 8 o ?3 + 4^2 4- 2ai 4- ao, 

960 = 16ûf4 — 8o?3 4- 4af2 — 2a\ 4- ao, 

59,235 = 81a 4 4- 27a3 4- 9a 2 + 3ai 4- ao- 


9 -9] [6 -9] [-50,801 -56,632" 

3 -3_|‘ 2U * [3 —6 J* 2A * |_ 113,264 119,095 ‘ 


3,007 -5,120] [ 938 160" 

1,024 -3,067j' L- 32 1130 ‘ 



25. 2, 569 = 4a2 4- 2a 1 4- ao, —339 —766 1110 

5, 633 = 4a 2 — 2ai 4- ao, —4440 4101 344 . 

5= «2 4- û'i+û'o- —1376 —3064 4445 


26. 0.814453 = 0.25a 2 4- 0.5ai 4- a 0 , [1.045578 0.003906 -0.932312" 

0.810547 = 0.25^2 — 0.5ai + a 0 , 0.058270 0.812500 -0.229172 . 

0.988285 = 0. 0625a 2 4- 0.25ai + a 0 . |_0.014323 0.000977 0.755207 


Section 7.4 

1. 128 = 2a 1 4- ao, 

448 = a\. 

3. 128 = 4^2 4- 2a 1 4- û'o, 

448 = 4û?2 + a 1 , 

1 = a 2 4- ai 4- ao. 

5. 59,049 = 9a2 4- 3ai 4- ao, 

196,830 = 6a 2 4- ai, 
590,490 = 2a 2 . 


7. 512 = 8 a 3 4 - 4a 2 4- 2ai 4- ao, 

2,304 = 12a3 4- 4a 2 4- ai, 
9,216 = 12a3 4- 2a 2 , 

32,256 = 6a3. 

9. 512 = 8 a 3 4 - 4^2 4- 2a 1 4- ao, 

2,304 = 12 a 3 4 - 4^2 4- ai, 

1 = a3 4- a 2 4- ai 4- ao. 
9 = 3a3 4 - 2a 2 4- ai. 


2. 128 = 4a 2 4- 2a 1 4- ao, 

448 = 4a 2 4- ai, 

1,344 = 2a 2 . 

4. 59,049 = 3ai 4- ao, 

196,830 = ai. 


6 . 59,049 = 27a3 4 - 9a 2 4- 3ai 4- ao, 

196,830 = 27a3 + 6a 2 4- ai, 
590,490 = 18a3 4- 2a 2 , 

1,574,640 = 6a 3 . 

8 . 512 = 8 a 3 4 - 4a 2 4- 2ai + ao, 

2,304 = 12a3 4- 4a 2 4- ai, 

9,216 = 12a3 4- 2a 2 , 

1 = a3 4- 0 L 2 4- ai 4- ao. 
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10. (5 ) 10 - 3(5 ) 5 = as(5 ) 5 + a 4 ( 5) 4 + a 3 ( 5) 3 + a 2 (5 ) 2 + ai (5) + ao 

10(5 ) 9 — 15(5) 4 = 5 ûf 5 ( 5) 4 + 4 û' 4 ( 5) 3 + 3 a 3 ( 5) 2 + 2a 2 (5) 4- ai, 

90(5) 8 — 60(5) 3 = 20 a 5 ( 5) 3 + 12 a 4 ( 5) 2 + 6 ^ 3 ( 5 ) + 2a 2 , 

720(5) 7 — 180(5) 2 = 60û'5(5) 2 H- 24^4(5) 4- 60 ^ 3 , 

(2 ) 10 - 3(2 ) 5 = a 5 (2 ) 5 + a 4 (2 ) 4 + a 3 (2 ) 3 + a 2 (2 ) 2 4- ai (2) + ao 

10(2 ) 9 — 15(2) 4 = 5a 5 (2 ) 4 + 4a 4 (2 ) 3 + 3a 3 (2 ) 2 + 2a 2 (2) + «i. 


11 . 

"729 

0 

0 ' 
729 

12 . 

“4 

0 

1 

-1 

-3 

0 

13. 

"0 

0 

0 

0 

0 " 

0 



5 

1 

-4 


0 

0 

0 


Section 7.5 

1. e = ai + ao, 2. e 2 = 2a\ + c^n, 3. e 2 = 4a2 + 2a 1 + an, 


e 2 — 2a 1 + ao. e 2 — a\. 

4. e 1 = a 2 + ai + ao, 
e -2 = 4a 2 — 2a 1 + ao, 
e 3 = 9a 2 + 3ai + ao- 

6. sin (1) = a 2 + ai+ao, 
sin (2) = 4a 2 + 2ai + ao, 
sin (3) = 9a 2 + 3ai + ao- 

8. e 2 — 8a 3 + 4 a 2 + 2 ai + ao, 

e = 12a3 + 4 a 2 + ai, 
e 2 = 12a3 + 2a 2 , 
e 2 = 6a3. 


e 2 = 4a 2 4 - ai, 
e 2 — 2a 2 . 

5. e~ 2 = 4a 2 — 2a 1 + ao, 

e~ 2 — — 4a 2 4- ai, 
e 1 = a 2 4- ai 4- ao- 

7. sin (—2) = 4a 2 — 2a 1 4- ao, 

cos (—2) = — 4a 2 4 - ai, 
sin(l)= a 2 + ai+ao- 

9. e 2 — 8 a 3 + 4a 2 + 2ai + ao, 
e — 12 a 3 4- 4a 2 4- ai, 
e~ 2 — — 8 a 3 4- 4a 2 — 2a 1 4- ao, 
e~ 2 — 12 a 3 — 4a 2 4- ai. 


10. sin (2) = 8 a 3 4- 4a 2 4- 2a 1 + ao, 
cos (2) = 12 a 3 4 - 4a 2 4- ai, 

sin (—2) = — 8 a 3 4- 4a 2 — 2ai 4- ao, 
cos (—2) = 12 a 3 — 4a 2 4- ai. 

11. e 3 = 27a3 + 9a 2 + 3ai 4- ao, 
e 3 = 27a3 + 6a 2 + ai, 

e 3 = 18 a 3 + 2 a 2 , 
e~ l = — a 3 + a 2 — ai + ao. 


12. cos (3) = 27a3 + 9a 2 4 - 3a 1 + ao, 

— sin (3) = 27a3 4 - 6a 2 4- ai , 

— cos (3) = 18a3 4 - 2a 2 , 

cos (—1) = — a 3 4- a 2 — ai + ao. 
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13. 


15. 


1 

7 


'3e 5 + 4e “ 2 3e 5 - 
_4e 5 - 4e -2 4e 5 + 


0 13' 
-12 5 
0 0 1 


3e- 2 ' 

3e“ 2 


14. e 


'2 - 1 ' 

1 0 


16. 


1 

Ï6 


12e 2 + 4e -2 4e 2 

12e 2 - 12e -2 4e 2 

0 


4e -2 
+ 12e -2 
0 


38e 2 + 2e -2 ' 
46e 2 - 6e -2 
16e 2 


17. I 

5 


-1 6 

4 1 


18. (a) 


l°g(3/2) log(3/2) - log(l/2) 
0 log(l/2) 


(b) and (c) are not defined since they possess eigenvalues having absolute 
value greater than 1. 


(d) 


0 0' 

0 0 


Section 7.6 


1. 1/7 


3e 8 ' + 4 et 
3e 8 ' - 3e' 


4e 8 ' _ 4e r 
4e 8r + 3e f ' 


(2/s/3) sinh V3t + cosh V3r 
(—1 />/3) sinh V3 ~t 


(1//3) sinh s/3 ~t 
(— 2/s/3) sinh V3 1 + cosh V3r 


Note: 

, e '/3t _ -\/3 r 

sinh V3t = J- 


and 


cosh V3 1 — 





1.4e _2r - 0.4e~ 7 ' 
—2.8e~ 2 ' + 2.8e~ 11 


0.2e- 2 ' - 0.2e _7r 
— 0.4e _2r + 1.4e _7r ' 



0.8e _2r + 0.2e -7 ' 
0.4e _2r - 0.4e- 7 ' 


0.4e -2 ' - 0.4e -7 '" 
0.2e“ 2 ' + 0.8e- 7 ' ' 



0.5e -4 ' + 0.5e- 16 ' 
0.5e -4 ' - 0.5e- 16 ' 


0.5e -4 ' - 0.5e -16 '" 
0.5e“ 4 ' + 0.5e- 16 ' ' 


1 t t 2 / 2 
0 1 t 

0 0 1 


7 . 
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8 . 


1 

12 


12e* 

-9e' + 14e 3 ' - 5e~ 3 ' 
-24e' + 14e 3 ' + 10e~ 3 ' 


0 0 

Se 3 ' + 4e~ 3 ' 4e 3 ' - 4e~ 3 ' 

4.f fi — 'Xt a 'X 1 


8e 3 ’ 


H-e — L te 

8e~ 3t 4e 3t + 8é>~ 3r _ 


Section 7.7 


1. 


(1 /2) sin 2 t + cos 2 1 (—1 /2) sin 2 t 

(5/2)sin2 t (— 1/2) sin2r + cos2r 


2 . 


3. 


\/2 sin V2/- + cos \/2Â 

(3/V5) sin 

cos (80 ^ sin (80 

—8 sin (80 cos (80 


— V2sin V2/- 
V2sin V2r + cos y/2 1 


4. 


4 


2 sin (80 4- 4 cos (80 
5 sin (80 


-4 sin (80 

■2 sin (80 4- 4 cos (80 


5. 


2 sin(0 4- cos(0 5 sin(0 
— sin(0 —2 sin(0 4- cos(0 


6. -e 4t 
3 


7. g 


8 . 


4r 

1 

0 

0 


4 sin(30 4- 3 cos(30 sin (30 

—25 sin (30 —4 sin(30 4- 3 cos(30 


sin 1 4- cos t sin t 
—2 sin f sin f + cos t 

-2 + 2 cos(0 + sin(0 —5 + 5 cos(0 
cos(0 — 2 sin(0 —5 sin(0 

sin(0 cos(0+2sin(0 


Section 7.8 

3. A does not hâve an inverse. 

e 2e 

0 


8. e x = 


e 

0 


e — 1 
1 




1 e-1 
0 1 


e A e R 


9. 

11. 


e R e A — 


'e 2e - 2' 

.a+b _ 

e 2e 

0 e 

•> e 

0 


A = 


1 0 ' 
0 2 


B = 


3 0' 
0 4 


Also see Problem 10. 


First show that for any integer n, (P ^P)" = P 1 B /1 P, 
directly. 


— 2e 
e 


and then use Eq. (6) 
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Section 7.9 

6 t 2 e r 2* — 1 

2t + 3 2 cos 2* 

— 18* cos 2 (3* 2 ) sin(3* 2 ) 0 

sin* + ci ^* 3 — t + c*2 

* 2 + C3 + C4 


1. (a) 


— sin * 
2 


2 * 

,{t- 1) 


(b) 


CHAPTER 8 


Section 8.1 


1. x(*) = 


2. x(*) = 


3. x(*) = 


4. x(*) = 


5. x(*) = 


6. x(*) — 


7. x(*) = 


8. x(*) = 


x(t) 
y(t ) 

7(0 

^(0 

*(0 

*(0 

7(0 

*(0 

7(0. 

u{t) 

v(t) 

_w(t) 

x(t) 

7(0 

_z(0. 

"K0‘ 

KO 

u(t) 


, A(0 = 


, A(0 = 


, A(*) = 


, A(0 = 


, A(*) = 


'2 3' 
4 5 

'3 2' 
4 1 


, f (0 = 


, f (0 = 


0 

0 

0 

0 


, C = 


■3 3' 

4 -4 


, f (0 = 


, C = 

1 

-1 


' 6 ' 

7 

T 

1 


, *o = 0. 


, * 0 = 0. 


, C = 


3 0' 
2 0 


, f (0 = 


* 

* + 1 


, c = 


0' 

0 

1 

1 


, *0 = 0 


, *o = 0 


' 3 1 2 i 
1 * 


, f (0 = 


' 2 ' 

2t 


c = 


2 ' 

■3 


, *o = l 



e 1 

* 

1 




0 



, A (t) = 

* 2 

-3 

* + 1 

, m 

— 

0 

, c 

— 

— 

_0 

1 

e v 

z 



0 


— 

-1 

"0 

6 

r 



"0" 

1 


"10" 

, Mt) = 

1 

0 

-3 

, Ut) = 

0 

, C = 

10 


0 

-2 

0 



0 



20 


C 
1 
-1 


, A(*) = 


1 

2 


—3 —sin* 
-1 0 
e l * 2 - 1 


, f (0 = 


sin* 
* 2 -l 
cos * 


c = 


4 
■2 

5 


, *o = l 


9. Only (c). 10. Only (c). 11. Only (b). 


0 

1 

1 / 0 . 


, *0 — 4. 


*o = 0. 
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Section 8.2 


1. x(0 = 


*i(0 

Xlit) 




2 . x(t) — 


xi (0 
■*2(0 




3. x(0 = 


xi (0 

*2(0 J’ 




4. x(0 = 


xi (0 
*2(0 



1 

2e r 





5. x(0 = 


xi (0 
* 2(0 





*1 (0 


" 0 

1 

0 


" 0 " 


2" 

6. x(0 = 

*2(0 

, A(0 = 

0 

0 

1 

, f(0 = 

0 

, C = 

1 


*3 (0 


1/4 

0 

-f/4 


0 


-205 


t{) — — 1 . 


7. x(0 = 


xi (0 
* 2(0 
X3 (0 

*4(0 



1 

0 

0 


= 0. 


0 

1 

0 


0 

0 

1 

0 



1 

2 

JT 



X] (?) 


"0 

1 

0 

0 

0 

0 


0 


*2(0 


0 

0 

1 

0 

0 

0 


0 

8. x(0 = 

X^(t) 

, A(0 = 

0 

0 

0 

1 

0 

0 

, f(0 = 

0 

V / 

*4(0 

0 

0 

0 

0 

1 

0 

0 


*5 (0 


0 

0 

0 

0 

0 

1 


0 


*6(0 


0 

0 

0 

0 

-4 

0 


t 2 - 1 


2 

1 

0 

2 

1 

0 
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Section 8.3 



3. x(0 = yi(t) , AO) = 0 

[_ t 2 







1 0 

1 0 

0 0 

-1 0 


0 t 2 
0 1 
t 0 



1 0 
2 0 
0 0 
0 0 
0 0 
0 -t 


1 0 
0 1 
0 0 
0 0 
0 1 




0 0 
0 0 
1 0 
0 1 
0 0 
0 1 



o oi ro" 

0 0 0 

-i i , m = o , 

0 1 0 

o 2 ] [o 
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Xï(t) 


"0 

1 

0 

0 


"-2 


" 1 

7. x(t) = 

y\ (0 
yi (0 

, A(r) = 

0 

0 

0 

0 

1 

0 

0 

1 

, HO = 

0 

-2 

, C = 

2 

17 


_zi(0_ 


1 

1 

0 

0 


0 


0 



Xi (f) 


"0 

1 

0 

0 

0 

0 


0" 


■*2(0 


0 

0 

1 

0 

1 

0 


2 

8 . x(0 = 

yi(t) 

MO 

, A(r) = 

0 

1 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

, HO = 

0 

-1 


Zl(t) 


0 

0 

0 

0 

0 

1 


0 


_Z2(0_ 


1 

0 

0 

0 

-1 

0 


1 



Section 8.4 


3. (a) e“ 3r 

"1 

-t 

t 1 2 /2 

(b) e 3(r “ 2) 

”1 {t- 2) 

0 f - 2)2/2" 

0 

1 

-t 

0 

1 

( t - 2) 


0 

0 

1 


0 

0 

1 


(c) e ^ ^ 


1 

0 

0 


(t- 2) 
1 
0 


(r - s) 2 / 2 
C t-s ) 

1 


(d) e 2) 


1 

0 

0 


-i.t-2) 

1 

0 


0 t - 2)2/2 
-(r - j) 

1 




1 r 2e — 5r 

6 4e~ 5r 


+ 4e' 
— 4e f 


2e~ 5 ' - 2e r 
4e -5 ' + 2e r " 



1 T 2e“ 5 " + 4e ç 
6 4e“ 5 ' s -4e s 


2e“ 5 * - 2e r 
4e _5i + 2e s ’ 



1 |"2^ 5(r_3) + 4e - ^ -3 ) 2^ 5(r_3) — 

5 4^5(r-3) _ 4^-(f-3) 4^5(r-3) _|_ 2e~^~^ ’ 




1 sin 3r + 3 cos 3r 
3 2 sin 3 t 


—5 sin3^ 

— sin 3^ + 3 cos 3f ’ 



1 sin 3s + 3 cos 3s 

3 2 sin 3s 


—5 sin 3s 

— sin 3s + 3 cos 3s ’ 
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sin3(Y — s) + 3 cos 3(7 — s) — 5sin3 (t — s) 

2sin3 (t — s) — sin3(> — s) + 3 cos3(Y — s)t 

7. x(t) = 5e^- 2) - 3e~ ( ' t ~ 2 \ y(t) = 5e ( '~ 2) - ^- (r “ 2) . 

8. x(t) = 2e<'“ 1 > - 1, y(t) = 2e^ - 1. 

9. x(t) — + 3&4£ -r , y(t) = k^e' + k 4 e ~ t . 

10 . x(t) — k^e 1 + 3k 4 e- 1 — 1 , y(t) = k^e 1 + k^e -1 — 1 . 

11. x(0 = cos 2t — (1 /6) sin 2 1 + (1 /3) sin 

12. *(f) = r 4 /24 + (5/4)/ l2 - (2/3 )t + 3/8. 

13. x(t) = (4/9) e 2 ' + (5/9) e~ h - (1/3) te~ lt 

14. x(t) — —8 cos t — 6 sin r + 8 + 6t, 
y(t ) = 4 cos t — 2 sin £ — 3. 



Section 8.5 


4. First show that 


<ï> T (fl,fo) 


h 


-.-1 


4>(^l, s)ds 


Ut 0 




4>(*o, T) [ s)&(ti,s)ds* T (to, h) 
Jt 0 

L 


-.-1 


i — 1 


*(fo, *)[<K'o, *!)$(*!, ■*)] * 


CHAPTER 9 

Section 9.1 

1. (a) The English alphabet: a, b,c, . . . x, y , z. 26. 5/26. 

(b) The 366 days designated by a 2008 Calendar, ranging from 1 January 
through 31 December. 366. 1/366. 

(c) A list of ail 43 United States Presidents. 43. 1/43. 

(d) Same as (c). 43. 2/43 (Grover Cleveland was both the 22 nd and 24 th 
President). 

(e) Regular deck of 52 cards. 52. 1/52. 

(f) Pinochle deck of 48 cards. 48. 2/48. 

(g) See Figure 9.1 of Chapter 9. 36. 1/36. 



402 


Answers and Hints to Selected Problems 


(h) Same as (g). 

(j) Same as (g). 2/36. 
(1) Same as (g). 7/36. 

(n) Same as (g). 12/36. 


(i) Same as (g). 5/36. 
(k) Same as (g). 18/36. 
(m) Same as (g). 5/36. 

(o) Same as (n). 


(p) Same as (g). 0. 

2. The sample space would consist of ail 216 possibilities, ranging from rolling a 
“3” to tossing an “18”. 

3. 90. 4. 1950. 


Section 9.2 



1. (a) 8/52. 

(b) 16/52. 

(c) 28/52. 

(d) 2/52. 

(e) 28/52. 

(f) 26/52. 

(g) 39/52. 

(h) 48/52. 

(i) 36/52. 

2. (a) 18/36. 

(b) 15/36. 

(c) 10/36. 

(d) 30/36. 

(e) 26/36. 

(f) 1. 

3. (a) 108/216. 

(b) 1/216. 

(c) 1/216. 

(d) 3/216. 

(e) 3/216. 

(f) 0. 

(g) 213/216. 

(h) 210/216. 

(i) 206/216. 


4. 0.75. 5. 0.4. 


6. P(A U B U C U D) = P(A) + P(fl) + P(C) + P(D) - P(A n fl) - 
P(A n C) - P(A n D) - P(B n C) - P(B n D) - p(c n D) + 

P(A n fl n C) + P(A n fl n D) + P(A n c n D) + 

P(fl n c n D) - P(A n fl n c n D). 


Section 9.3 




1. (a) 15. 

(b) 7. 

(c) 56. 

(d) 190. 

(e) 190. 

(f) 1. 

(g) i. 

(h) 100. 

(i) 1000. 

(j) 1. 



2. 2,042,975. 

3. 5005. 




4. (a) Approximately .372. (b) Approximately .104. 

(c) Approximately .135. (d) Approximately .969. 

(e) Approximately .767. 
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5. (a) ( ^3 ) (-65) 123 (.35) 377 

(b) ( J™ ) (,65) 485 (.35) 15 . 

\ 

(c) ( 4^) (- 65 ) 497 (- 35 ) 3 + (49g ) (- 65 ) 498 (- 35 ) 2 


+ ( aqq) (-65) 4 " (-35) 1 + ( r™) (.65) 500 (.35)° 


500' 


499 


500 


(d) 1 - P (.65)“ (*35) 2 - P (.65)“ (.35)' - (*» ) (.65)” (.35)“. 

(e) P (.65)'»» (.35)“ + P (.65)“ (.35)“ + P (.65)” (.35)“ 

/500\ / .^400 / oc\ioo . /500\ / ,-x500 / 

+ (^400 ) ^ v*^) (^500 ) (' 65 ) (-35) • 

6 . Approximately .267. 

7. Approximately .267. 


Section 9.4 

1. (a) There is a négative element in the second row. 

(b) The first row does not add to 1. 

(c) The third row does not add to 1. 

(d) It is not a square matrix. 

2. (a) If it is sunny today, there is a probability of .5 that it will be sunny tomor- 

row and a .5 probability that it will rain tomorrow. If it rains today, there is 
a .7 probability that it will be sunny tomorrow and a .3 chance that it will 
rain tomorrow. 

(b) If a parking meter works today, there is a probability of .95 that it will 
work tomorrow with a .05 probability that it will not work tomorrow. If the 
parking meter is inoperative today, there is a probability of .02 that it will 
be fixed tomorrow and a .98 probability that it will not be fixed tomorrow. 

(c) Any scénario has a “50-50” chance at any stage. 

(d) What is “good” stays “good”; what is “bad” stays “bad”. 

(e) What is “good” today is “bad” tomorrow; what is “bad” today is “good” 
tomorrow. 
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(f) See Example 2 in Section 9.4 and use Tinker, Evers, and Chance for Moe, 
Curly, and Larry and instead of visiting or staying home use “borrowing a 
car” or “not borrowing a car”. 


3. Clearly if we raise either matrix to any power, we obtain the original matrix. 


1 0 

4. The even powers produce j and the odd powers give back the original 
matrix. And situation repeats itself after an even number of time periods. 

5. p® = 0.4, P 21 = 0.15, p ( H = 0.7, p^ = 0.825. 

6. (a) 


.7 .3 
.4 .6 


(b) Approximately 0.5725. 

(c) Approximately 0.5717. 


CHAPTER 10 

Section 10.1 

1. 11,5. 

5. 64,68. 

9. 5/6, 7/18. 

13. 2,3. 


2. 8,4. 

6. 6,5. 

10 . 5/V6, 1. 
14. 1,1. 


17. undefined, 6. 18. 


'3/5' 

4/5 


3. -50,74. 

7. 26,24. 

11. 7/24, 1/3. 
15. -19,147. 


19. 


4/V41 

L-5/V4ÏJ- 


4. 63,205. 

8. -30,38. 
12. 0,1400. 

16. -1/5, 1/5. 


20 . 


1/V58 3/V58 



-4/V34 


3/vT7 _ 



21. 

3/V34 

22. 

-2/vT7 

. 23. 

2/V2Ï 4/V2Î 1/V2Î 


-3/V34 


L— 2/VI7J 




24. 


4/VÏ9Ï 
-6/VÏ97 
-9/VÏ97 
L 8/VÎ97J 


25. 


1/V55 2/V55 -3V55 4/V55 -5/V55 


26 


-3/V259 8/V259 11/V259 - 4/V259 7/V259 . 


27. No vector x exists. 28. Yes, see Problem 12. 

33. ||x + y || 2 = (x + y, x + y} = (x, x> + 2(x, y> + (y, y> 

= ll x ll 2 + 2{x, y) + ||y|| 2 . 

34. Show that ||x — y|| 2 = ||x|| 2 — 2(x, y> + ||y|| 2 , and then use Problem 33. 
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37. Note that (x, y} < |(x, y)|. 38. (x, y} = det(x T y). 

40. 145 41. 27. 42. 32. 


Section 10.2 


1. x and y, x and u, y and y, u and v. 

2. x and z, x and u, y and u, z and u, y and y. 


3. -20/3. 


4. -4. 


5. 0.5. 


6. x — — 3y. 


7. x = 1, y = —2. 8. x = y = — z. 9. x = y = — z; z = d=l/V3. 


10 . 


1/V5 



2/V5' 

9 

1/V5_ 


11 . 


12 . 


14. 


16. 


3/VÏ3 
-2/VÏ3 J 


"2/V5 

1/V5 

0 


' 0 

3/5 

4/5 


2/V\3 
' |_3/\/Ï3 

-2/V45 

4/V45 


13. 



1/3" 

9 

-2/3 


2/3 


15. 


1/V2 


'1/V6 

2/V6 

.1/V6 

'1/V2 
1/V2 
0 



'-1/V2' 

9 

1/V2. 



' 1/V3" 


"-1/V2" 

9 

— 1/V3 

9 

0 

— 1 

1/V3_ 


1/V2_ 

— 1 

1/V3" 


"-1/V6" 

9 

-1/V3 

9 

1/V6 


1/V3_ 


2/V6_ 



3/5 


4/5 

9 

16/25 

9 

-12/25 


-12/25 


9/25 



" 0 " 


' 3/VÏ5" 


" 3/V35" 



1/V7" 

17. 

1/V3 


-2/VÎ5 


3/V35 



1 / 9/7 

1/V3 

9 

1/VÏ5 

9 

-4/V35 

9 


1/V7 


_1/V3_ 


_ 1/VÏ5. 


L 1/V35_ 


— 

-2/V7_ 


"1/V2" 


"-1/V6" 


' 1/V3 " 


— 

0" 


18. 

1/V2 


1/V6 


-1/V3 



0 


0 

9 

-2/V6 

9 

-1/V3 

9 


0 



0 


0 


0 


— 

1 



23. ||x - y || 2 = {x - y, x - y) = ||x|| 2 - 2(x, y> + ||y|| 2 . 

24. || sx + fy|| 2 = (sx - /y, sx - ry> = ||^x|| 2 - 2st(x, y) + ||ry|| 2 . 

25. I. 26. Set y = x and use Property (II) of Section 10.1. 

28. Dénoté the columns of A as Ai, A 2 , . . . , A n , and the éléments of y as 

yi,yi> respectively. Then, Ay = Aiyi + A 2 y 2 + bA„y„ and 

(Ay, p) = yi(Ai, p> + y 2 (A 2 , p> + • • • + y n { A„, p). 
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Section 10. B 


1. (a) 0 = 36.9°, 


2. (a) 0 = 14.0°, 


3. (a) 0 = 78.7°, 


4. (a) 0 = 90°, 


5. (a) 0= 118.5°, 


6. (a) 0 = 50.8°, 


7. (a) 0 = 19.5°, 


8. (a) 0 = 17.7°, 


9. (a) 0 = 48.2°, 


10. (a) 0 = 121.4°, 


11 . 


12 . 


13 . 


0.4472 0.8944' 

0.8944 -0.4472 

0.7071 -0.7071' 
0.7071 0.7071 

0.8321 0.5547' 
-0.5547 0.8321 


(b) 


(b) 


(b) 


(b) 


(b) 


1 . 6 ' 

0.8 


0.7059' 

1.1765 


0.5' 

0.5 


0 

0 


■0.7529' 

-3.3882 


(b) 


(b) 


(b) 


(b) 


(b) 


T 

0 

1 


'8/9 

8/9 

4/9 


'1.2963' 

3.2407 

3.2407 


'2/3' 

2/3 

2/3 

0 


'-7/6' 

7/3 

0 

7/6 


'2.2361 1.7889' 
0.0000 1.3416 

'1.4142 5.6569' 
0.0000 1.4142 

'3.6056 0.8321' 
0.0000 4.1603 


(c) 


(c) 


(c) 


(c) 


(c) 


(c) 


(c) 


(c) 


(c) 


(c) 


- 0 . 6 ' 

1.2 

0.2941' 

-0.1765 

2.5' 
-2.5 

4 

-1 

-6.2471' 

1.3882 

1 ' 

1 

-1 

1/9' 
1/9 
-4/9 

'-1.2963' 

-0.2407 

0.7593 

'-2/3' 
1/3 
1/3 

1 

'13/6' 
-1/3 
3 

17/6 
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14. 


15. 


20 . 


21 . 


' 0.3333 

0.6667 

0.6667 

' 0.3015 

0.3015 

0.9045 


0 . 8085 ' 

0.1617 

- 0.5659 

- 0 . 2752 ' 

- 0.8808 

0.3853 


' 3.0000 2 . 6667 ' 
0.0000 1.3744 


' 3.3166 4 . 8242 ' 
0.0000 1.6514 



0.7746 

0 . 4034 " 






16. 

- 0.5164 

0.5714 


" 3.8730 

0 . 2582 " 




0.2582 

0.4706 


0.0000 

1 . 

9833 

• 




- 0.2582 

0.5378 







" 0.8944 

- 0.2981 

0 . 3333 " 


" 2.2361 

0.4472 

1 . 7889 " 

17. 

0.4472 

0.5963 

- 0.6667 


0.0000 

1.3416 

0.8944 


0.0000 

0.7454 

0.6667 


0.0000 

0.0000 

2.0000 


" 0.7071 

0.5774 

- 0 . 4082 " 


" 1.4142 

1.4142 

2 . 8284 " 

18. 

0.7071 

- 0.5774 

0.4082 


0.0000 

1.7321 

0.5774 


0.0000 

0.5774 

0.8165 


0.0000 

0.0000 

0.8165 



L_ 

_J L_ 


" 0.00 0.60 0.80 


"5 3 7 " 

19. 

0.60 0.64 - 0.48 


0 5 2 


0.80 - 0.48 0.36 


0 0 1 


' 0.0000 

0.5774 

0.5774 

0.5774 

' 0.7071 

0.7071 

0.0000 

0.0000 


0.7746 

0.5164 

0.2582 

0.2582 

0.4082 

0.4082 

- 0.8165 

0.0000 


0 . 5071 ' 

0.5071 

- 0.6761 

0.1690 

0 . 5774 ' 

- 0.5774 

- 0.5774 

0.0000 


' 1.7321 

0.0000 

0.0000 


' 1.4142 

0.0000 

0.0000 


1.1547 

1.2910 

0.0000 


0.7071 

1.2247 

0.0000 


1 . 1547 ' 

0.5164 

1.1832 


0 . 7071 ' 

0.4082 

1.1547 


24. QR £ A. 


Section 10.4 
1. Ai = RoQo + 71 


" 19.3132 

- 1.2945 

0.0000 


"— 0.3624 

0.0756 

0 . 9289 " 

0.0000 

7.0231 

- 0.9967 


0.0000 

- 0.9967 

0.0811 

0.0000 

0.0000 

0.0811 


0.9320 

0.0294 

0.3613 



"1 

0 

0 " 


0.0000 

2.7499 

17 . 8357 " 

+ 7 

0 

1 

0 

— 

- 0.9289 

- 0.0293 

0.2095 


0 

0 

1 


0.0756 

0.0024 

7.0293 
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2. Ai = RqQo - 141 


"24.3721 

-17.8483 

3.8979” 


0.6565 

-0.6250 

0.4223” 

0.0000 

8.4522 

-4.6650 


-0.6975 

-0.2898 

0.6553 

0.0000 

0.0000 

3.6117 


0.2872 

0.7248 

0.6262 



"1 

0 

0 


” 15.5690 

-7.2354 

1.0373" 

14 

0 

1 

0 

— 

-7.2354 

-19.8307 

2.6178 


0 

0 

1 


1.0373 

2.6178 

-11.7383 


3. Shift by 4. 



4.1231 

0.0000 

0.0000 

0.0000 


0.9701 

4.0073 

0.0000 

0.0000 


0.0000 

-0.9982 

4.0005 

0.0000 


13.5820 

-4.1982 

12.9509 

3.3435 



-0.9701 

0.2425 

0.0000 

0.0000 


-0.2349 -0.0586 -0.0151 
-0.9395 -0.2344 -0.0605 
0.2495 -0.9376 -0.2421 ' 
0.0000 0.2500 -0.9683 


Ai = RoQo + 41 = 


-0.2353 -0.0570 
0.9719 -0.0138 
0.0000 0.9983 

0.0000 0.0000 


3.3809 -13.1545 
-1.0529 4.0640 

3.4864 -13.5081 ' 
0.8358 0.7626 


4. 7.2077, -0.1039 ±1.5769/. 5. -11,-22,17. 6. 2,3,9. 

7. Method fails. Ao — 71 does not hâve linearly independent columns, so no QR- 
decomposition is possible. 

8.2,2,16. 9. 1,3,3. 10. 2, 3 ± /. 11.1 , ±/. 

12. ± /, 2 ± 3 /. 13. 3.1265 ± 1.2638/, - 2.6265 ± 0.7590/. 

14. 0.0102,0.8431,3.8581,30.887. 


Section 10.5 


1. 

3. 

5. 

7. 

9. 

11. 


x = 2.225, y = 1.464. 2. 

v = 9.879, y = 18.398. 4. 

x = 1.512, y = 0.639, z = 0.945. 6. 

x = 81.003, y = 50.870, z = 38.801. 8. 

2 and 4. 10. 


x = 3.171, y = 2.286. 
x = -1.174, y = 8.105. 
x = 7.845, y = 1.548, z = 5.190. 
x = 2.818, y = —0.364, z = —1.364. 
(b) y = 2.3* ± 8.1, (c) 21.9. 


(b) y = —2.6* ± 54.4, (c) 31 in week 9, 28 in week 10. 
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12. (b) y = 0.27* + 10.24, (c) 12.4. 


13. m — 


N N N 


n E xm - E x i E yi 



C = 


N N 


T,y< E 

i=l i=l 



N N 

E x i E x iyi 

i = 1 i = 1 



If N J^iL i x } is near •*/ j » then the denominator is near zéro. 

14. £*, *; = o, so the denominator for m and c as suggested in Problem 13 is 
simply NJ2ti (*/) 2 - 


15. y = 2.3*' + 15. 16. y = -2.6*' + 42.9. 

17. (a) y = — 0.198*' + 21.18, (b) Year 2000 is coded as *' = 30; y (30) = 15.2. 

0.160" 

0.069 
-0.042 * 

-0.173 


23. E = 


0.841' 

0.210 

-2.312 


24. E = 



This page intentionally left blank 




Index 


A 

Adjacency matrices, 27-28, 42f 
Adjugates, 167-168 
Algorithms, 135 

Gram-Schmidt, revised, 331-337 
QR, 339-344 

Augmented matrix, 55-56 

B 

Bernoulli, Jakob, 305 
Bernoulli trials, 305-310 

binomial distribution and, 305 
independent events and, 305 
multiplication rule for, 305 
Binomial distribution, 305 
Normal Approximation to 

Binomial Distribution, 314 
Block diagonals, 31 
Bounded régions. See Finite régions; 
Régions 

c 

Calculations, for inversions, 

101-108 

theorems for, 101 

for square matrices, 102 
Calculus, for matrices, 213-255. 

See also Function e At 
Cayley-Hamilton theorem, 
219-222, 254-255 
conséquence of, 220 
vérification of, 219-220 
matrix dérivatives in, 248-254 
définitions of, 248-253 
properties of, 250-251 
polynomials in, 222-232 
for eigenvalues, 222-228 


for general cases, 228-233 
theorems for, 229-231 
well-defined functions in, 213-219, 
233-248 

définitions of, 214-216 
function e At , 238-248 
Maclaurin sériés as, 214 
matrix, 245 
scalar, 245 

Taylor sériés as, 216, 236 
theorems for, 216 
Cancellations, 14 
Cardinal numbers, 298 
Cauchy-Schwarz inequality, 319 
Cayley-Hamilton theorem, 219-222, 
254-255 

conséquence of, 220 
vérification of, 219-220 
Coding, 164 
Coefficient ratios, 137 
Cofactor matrices, 167 
Cofactors, expansion by, 152-156 
définitions of, 152-155 
minors and, 152 
pivotai condensation and, 165 
Column vectors, 33 
linearly dépendent, 349 
rank for, 78 
work, 143 

Combinations, in laws of probability, 
306-307 

Combinatorics, 305-310 

combinations v. permutations 
with, 306 

Commutativity, multiplication 
matrices and, 13 
Companion matrices, 184 


Complété pivoting strategies, for 
simultaneous linear 
équations, 69-71 
Gaussian élimination methods 
with, 69-70 

round-off errors and, 69 

Constant coefficient Systems, 
solution of, 275-286 
quantity replacement in, 
277-278 

Constraints. See Redundant 
constraints 

Cramer’s rule, 170-173 
restrictions on, 170-171 
simultaneous linear équations 
and, 173 


Décomposition. See LU 
décomposition 
DeMoivre’s formula, 241 
Dérivatives, for matrices, 248-254 
définitions for, 248-253 
properties of, 250-251 
Déterminants, 149-175 

cofactor expansion and, 152-156 
définitions of, 152-155 
minors and, 152 
pivotai expansion and, 165 
Cramer’s rule and, 170-173 
restrictions on, 170-171 
simultaneous linear équations 
and, 173 

définitions of, 149-150 
eigenvalues/eigenvectors and, 193 
inversion and, 167-170 
adjugates and, 167-168 
cofactor matrices and, 167 
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Déterminants ( continued ) 
définitions of, 167-168 
theorems for, 167 
pivotai condensation and, 163-167 
coding for, 164 
cofactor expansion and, 165 
properties of, 157-163 

multiplications of matrices and, 
158 

row vectors as, 157-158 
Diagonal éléments, 2 
Diagonal matrices, 21-22 
block, 31 

partitioning and, 31 
Disjoint events, 302 
compléments to, 309 
independent v., 305 
in Markov chains, 311 
Dominant eigenvalues, 202 
itérations of, 204-205 
Dot products, 327-330 

E 

Eigenvalues, 177-194, 201-212. 

See also Inverse power 
method 

for companion matrices, 184 
définitions of, 177-178 
dominant, 202 

itérations of, 204-205 
eigenvectors and, 180-190 
characteristic équation of, 
180-182 

function e At and, 241-244 
matrix calculus for, 222-228 
multiplicity of, 182 
power methods with, 201-212 
deficiencies of, 205 
inverse, 205-211 
properties of, 190-194 
déterminants and, 193 
trace of matrix as, 190-191 
for upper/lower triangular 
matrix, 191 

Eigenvectors, 177-212. See also 
Inverse power method 
for companion matrices, 184 
définitions of, 177-178 
eigenvalues and, 180-190 
characteristic équation of, 
180-182 

linearly independent, 186, 194-201 
theorems for, 195-200 
nontrivial solutions with, 185-186 


power methods with, 201-212 
deficiencies of, 205 
inverse, 205-211 
properties of, 190-194 
déterminants and, 193 
trace of matrix as, 190-191 
for upper/lower triangular 
matrix, 191 

Elementary matrices, inversions of, 
95-96 

Elementary row operations, 56 
pivots in, 59 
Eléments, 1-2 
diagonal, 2 

of multiplication matrices, 10-11 
Euler’s relations, 242-243 
Expansion by cofactors. See 

Cofactors, expansion by 

F 

Feasibility, région of, 131-132 
in linear programming, 

137-138 

minimization in, 132 
objective functions of, 132 
properties of, 131 
convex, 131 

Simplex method and, 140 
three-dimensional, 140 
unbounded, 133 
vertices in, 131 
Finite régions, 130 
First derived sets, 50 
Force, vectors and, 40 
Function e At , 238-248 

DeMoivre’s formula and, 241 
eigenvalues and, 241-244 
Euler’s relations and, 242-243 
properties of, 245-248 
Fundamental forms, 257-262 
définitions of, 259-261 
homogenous, 260 
initial value problems in, 258 
nonhomogenous, 260 
theorem for, 261 

Fundamental Theorem of Linear 
Programming, 135-136 

G 

Gaussian élimination method, 54-65 
augmented matrix in, 55-56 
with complété pivoting strategies, 
69-70 

définition of, 54-55 


elementary row operations in, 56 
pivots in, 59 

LU décomposition and, 121 
with partial pivoting strategies, 65 
Gram-Schmidt orthonormalization 
process, 322-325, 331-337 
for projections, 331-337 

revised algorithms for, 331-337 
Graphing, for inequalities, 127-131 
visualization for, 128 

H 

Hadley, G., 140 

Homogenous fundamental forms, 
260 

Horizontal bar, in Simplex method, 
142 


Identity matrices, 22 
inversions for, 102 
Independent events, 302, 305 
Bernoulli trials and, 305 
compléments to, 309 
disjoint v., 305 
in Markov chains, 311 
as sequential, 305 
Inequalities, 127-134. See also 
Feasibility, région of; 
Graphing 

Cauchy-Schwarz, 319 
finite régions and, 130 
graphing for, 127-131 
infinité régions and, 129 
intersections within, 129-130 
modeling with, 131-134 

région of feasibility and, 131 
strict, 129 

Infinité régions, 130 
Initial tableaux, 142 
Inner products, 315-344. See also 
Orthonormal vectors 
Cauchy-Schwarz inequality 
and, 319 
dot, 327-330 
nonzero vectors, 316 
normalized, 316 
orthonormal vectors, 320-327, 
334-344 
Gram-Schmidt 

orthonormalization process 
for, 322-325 
projections for, 327-338 
QR décompositions for, 334-344 
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sets of, 320 

theorems for, 321-325 
unit vectors, 316 

Intersections, within sets, 

129-130, 301 

Inverse power method, 205-211 
itérations of, 206t, 210t 
shifted, 209-210 

Inversions, of matrices, 7, 93-126 
calculations for, 101-108 
définition of, 93-95 
déterminants and, 167-170 
adjugates and, 167-168 
cofactor matrices and, 167 
définitions of, 167-168 
theorems for, 167 
for elementary matrices, 95-96 
for identity matrices, 102 
invertible, 93 
for lower triangular, 113 
LU décomposition in, 115-124 
construction of, 118-119 
Gaussian élimination methods 
and, 121 

for nonsingular square 
matrices, 115-116 
scalar négatives and, 117 
in upper triangular matrices, 
118 

nonsingular, 93 
properties of, 112-115 
extensions of, 113 
symmetry, 113 
theorems of, 112-113 
transpose, 113 

simultaneous équations and, 
109-112 
singular, 93 

for upper triangular, 113 

Invertible inversions, 93 

L 

Laws of probability, 301-307, 309 
combinations in, 306-307 
disjoint e vents under, 302 
independent events under, 302 
Bernoulli trials and, 305 
compléments to, 309 
disjoint v., 305 
as sequential, 305 

Least-squares, 344-354 
error in, 346 

linearly dépendent columns 
and, 349 


scatter diagram for, 344f, 345f 
straight line for, 347 
theorem for, 349-351 
Leontief closed models, 47^18 
Leontief Input-Output models, 

49-50 

Linear dependence, 72 
Linear differential équations, 

257-296. See also Solutions 
of Systems, with linear 
differential équations 
fundamental forms for, 257-262 
définitions of, 259-261 
homogenous, 260 
initial value problems in, 258 
nonhomogenous, 260 
theorem for, 261 
nlh order équations and, 
réduction of, 263-269 
variable définition for, 263 
solution of Systems with, 275-295 
with constant coefficients, 
275-286 

for general cases, 286-295 
System réduction in, 269-275 
Linear independence, 71-78 
définitions of, 71-73 
in eigenvectors, 186, 194-201 
theorems for, 195-200 
theorems for, 74-75 
vectors and, 71-76 
Linear Programming (Hadley), 140 
Linear programming, problem 
solving with, 135-140 
algorithms in, 135 
coefficient ratios in, 137 
Fundamental Theorem of Linear 
Programming, 135-136 
redundant constraints in, 137-138 
région of feasibility in, 137-138 
Linear Systems, 43-50 
définitions of, 43-45 

consistency as part of, 45 
homogeneity as part of, 45 
inconsistency as part of, 45 
nonhomogeneity as part of, 45 
solutions in, 43-44 
Leontief closed models for, 47-48 
Leontief Input-Output models for, 
49-50 

Linearly dépendent columns, 349 
Linearly independent eigenvectors, 
186, 194-201 
theorems for, 195-200 
Lower triangular matrices, 22 


eigenvalues/eigenvectors and, 191 
inversions of, 113 
LU décomposition, 115-124 
construction of, 118-119 
Gaussian élimination methods 
and, 121 

for nonsingular square matrices, 
115-116 

scalar négatives and, 117 
in upper triangular matrices, 118 

M 

Maclaurin sériés, 214 
Markov chains, 4-5 , 189-190,310-3 1 3 
modeling with, 310-313 
disjoint events in, 311 
independent events in, 311 
Mathematical models 
Leontief closed, 47-48 
Leontief Input-Output, 49-50 
Matrices, 1-40. See also Calculus, for 
matrices; Déterminants; 
Eigenvalues; Eigenvectors; 
Inner products; Inversions, of 
matrices; Multiplications, of 
matrices; Probability; 
Transpose matrices; Vectors 
augmented, 55-56 
basic concepts of, 1-5 
calculus for, 213-255 

Cayley-Hamilton theorem, 
219-222, 254-255 
matrix dérivatives in, 248-254 
polynomials in, 222-232 
well-defined functions in, 
213-219, 233-248 
cofactor, 167 
companion, 184 
constant coefficient Systems, 
solution of, 275-286 
quantity replacement in, 
277-278 

définition of, 1-2 
dérivatives of, 248-254 
définitions of, 248-253 
properties of, 250-251 
déterminants and, 149-175 

cofactor expansion and, 152-156 
Cramer’s rule and, 170-173 
définitions of, 149-150 
inversion and, 167-170 
pivotai condensation and, 
163-167 

properties of, 157-163 
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Matrices ( continuée i) 
diagonal, 21-22 
block, 31 

partitioning and, 31 
eigenvalues for, 177-194, 201-212 
companion matrices for, 184 
définitions of, 177-178 
dominant, 202 
eigenvectors and, 180-190 
multiplicity of, 182 
polynomials in, 222-228 
power methods with, 201-212 
properties of, 190-194 
eigenvectors for, 177-212 
companion matrices for, 184 
définitions of, 177-178 
eigenvalues and, 180-190 
linearly independent, 186, 
194-201 

nontrivial solutions with, 
185-186 

power methods with, 201-212 
properties of, 190-194 
elementary, 95-96 
inversions of, 95-96 
éléments in, 1-2 
equality between, 15 
inner products and, 315-344 
Cauchy-Schwarz inequality 
and, 319 
dot, 327-330 
nonzero vectors, 316 
orthonormal vectors, 320-327, 
334-344 
unit vectors, 316 
inversion of, 7, 93-126 
calculations for, 101-108 
définition of, 93-95 
déterminants and, 167-170 
for elementary matrices, 95-96 
for identity matrices, 102 
invertible, 93 
for lower triangular, 113 
LU décomposition in, 115-124 
nonsingular, 93 
properties of, 112-115 
simultaneous équations and, 
109-112 
singular, 93 

for upper triangular, 113 
least-squares and, 344-354 
error in, 346 

linearly dépendent columns 
and, 349 

scatter diagram for, 344f, 345f 


straight line for, 347 
theorem for, 349-351 
linear differential équations and, 
257-296 

fundamental forms for, 257-262 
nth order équations and, 
réduction of, 263-269 
solution of Systems with, 
275-295 

System réduction in, 269-275 
lower triangular, 22 
inversions of, 113 
operations of, 6-9 
définitions of, 6-7 
inversion in, 7 
multiplications in, 7, 9-19 
subtractions in, 7 
order of, 1 
partitioning, 29-32 

block diagonals and, 31 
définition of, 29 
polynomials in, 222-232 
for eigenvalues, 222-228 
for general cases, 228-233 
theorems for, 229-231 
probability and, 297-310 
Bernoulli trials and, 305-310 
combinatorics and, 305-310 
interprétations of, 297-298 
laws of, 301-307, 309 
Markov chains and, 4-5, 
189-190,310-313 
sets in, 297-300 
rank for, 78-82 
for columns, 78 
for rows, 78-82 
spécial, 19-29 

adjacency, 27-28, 42f 
diagonal, 2, 21-22 
identity, 22 
lower triangular, 22 
row-reduced form, 20-21 
skew symmetric, 22 
symmetric, 22 
transpose, 19-20 
upper triangular, 22 
square, 2 

inversions for, 102 
submatrices, 29-32 
zéro, 31 

transitional, 286-289 
upper triangular, 22 
inversions of, 113 
LU décomposition in, 118 
vectors and, 33-41 


column, 33 
components of, 33 
définitions, 33-34 
dimension of, 33 
geometry of, 37-41 
magnitude of, 33 
normalized, 34 
row, 33 
unit, 34 
zéro 

nonzero v., 20 
submatrices, 31 

Matrix calculus. See Calculus, for 
matrices 

Matrix functions, 245 
Minors, 152 

Models. See Mathematical models 
Multiplication rule, 305 
Multiplications, of matrices, 7, 9-19. 
See also Simultaneous linear 
équations 
cancellations in, 14 
coefficient matrix in, 14-15 
définitions of, 12, 15 
déterminants and, 158 
éléments of, 10-11 
partitioning and, 29 
properties of, 13 

commutativity and, 13 
rules of, 9-11 

postmultiplication, 10 
premultiplication, 10 
simultaneous linear équations 
and, 14, 43-91 
Cramer’s rule and, 170-173 
Gaussian élimination method 
in, 54-65 

linear independence and, 71-78 
linear Systems in, 43-50 
pivoting strategies for, 65-71 
rank and, 78-84 
substitution methods in, 50-54 
theory of solutions in, 84-88 


Négative numbers, location of, 143 
Nonhomogenous fundamental 
forms, 260 

Nonsingular inversions, 93 
Nonsingular square matrices, 
115-116 

Nontrivial solutions, 87 
with eigenvectors, 185-186 
Nonzero matrices, 20 
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Nonzero vectors, 316 
normalized, 316 

Normal Approximation to Binomial 
Distribution, 314 
Normalized vectors, 34 
n th order équations, 263-269 
réduction of, 263-269 
variable définition for, 263 


Optimization, 127-148. See also 
Simplex method, for 
optimization 
inequalities and, 127-134 
finite régions and, 130 
graphing for, 127-131 
infinité régions and, 129 
intersections within, 129-130 
modeling with, 131-134 
problems in, 132 
strict, 129 

linear programming and, 135-140 
algorithms in, 135 
coefficient ratios in, 137 
Fundamental Theorem of 
Linear Programming and, 
135-136 

redundant constraints in, 
137-138 

région of feasibility 
in, 137-138 

Simplex method for, 140-147 
horizontal bar in, 142 
initial tableaux in, 142 
in Linear Programming, 140 
région of feasibility and, 140 
slack variables in, 141 
steps in, 143 
vertical bar in, 142 
Orthogonal sets, 320 
Orthonormal vectors, 320-327 
Gram-Schmidt 

orthonormalization process 
for, 322-325, 331-337 
for projections, 331-337 
projections for, 327-338 
dot products in, 327-330 
Gram-Schmidt process for, 
331-337 

QR décompositions for, 334-344 
algorithms for, 339-344 
itérations of, 335-337 
sets of, 320 

theorems for, 321-325 


Partial pivoting strategies, for 
simultaneous linear 
équations, 65-67 
définition of, 65 
in Gaussian élimination 
methods, 65 
Partitioning, 29-32 

block diagonals and, 31 
définition of, 29 
multiplication of matrices 
and, 29 

Pivotai condensation, 163-167 
coding for, 164 
cofactor expansion and, 165 
Pivoting strategies, for simultaneous 
linear équations, 65-71 
complété, 69-71 

with Gaussian élimination 
methods, 69-70 
round-off errors and, 69 
partial, 65-67 
définition of, 65 
with Gaussian élimination 
methods, 65 
scaled, 67-68 
ratios in, 67-68 
Pivots, 59 

in work column location, 143 
Polynomials, 222-232 
for eigenvalues, 222-228 
for general cases, 228-233 
Postmultiplication, 10 
Power methods, with 

eigenvalues/eigenvectors, 

201-212 

deficiencies of, 205 
inverse, 205-211 

itérations of, 206t, 210t 
shifted, 209-210 
Premultiplication, 10 
Probability, 297-310. See also Laws 
of probability; Markov 
chains 

Bernoulli trials and, 305-310 
combinatorics and, 305-310 
interprétations of, 297-298 
laws of, 301-307, 309 

combinations in, 306-307 
disjoint events under, 302 
independent events under, 302 
Markov chains and, 4-5, 189-190, 
310-313 

modeling with, 310-313 


sets in, 297-300 

cardinal numbers in, 298 
Projections for, orthonormal vectors, 
327-338 

dot products in, 327-330 
Gram-Schmidt process for, 
331-337 

revised algorithms for, 331-337 

Q 

QR décompositions, 334-344 
algorithms for, 339-344 
itérations of, 335-337 
QR-algorithms, 339-344 

R 

Rank, in simultaneous linear 
équations, 78-84 
for column vectors, 78 
définition of, 78 
for row vectors, 78-82 
theorems for, 79, 82 
Ratios 

coefficient, 137 

in scaled pivoting strategies, 67-68 
Réduction of Systems, 269-275 
Redundant constraints, 137-138 
Région of feasibility. See Feasibility, 
région of 
Régions 
finite, 130 
infinité, 129 
Round-off errors, 69 
Row vectors, 33 

as déterminant property, 157-158 
rank for, 78-82 

Row-reduced form matrices, 20-21 
Ruth, Babe, 314 

s 

Scalar functions, 245 
Scalar négatives, 117 
Scaled pivoting strategies, for 
simultaneous linear 
équations, 67-68 
ratios in, 67-68 
Scatter diagram, 344f, 345f 
Sets, 297-300 

intersections within, 301 
orthogonal, 320 
orthonormal vectors in, 320 
union in, 297-301 
Shifted inverse power method, 
209-210 
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Simplex method, for optimization, 
140-147 

horizontal bar in, 142 
initial tableaux in, 142 
in Linear Programming, 140 
région of feasibility and, 140 
three-dimensional, 140 
slack variables in, 141 
steps in, 143 

négative number location, 143 
work column location, 143 
vertical bar in, 142 
Simultaneous linear équations, 14, 
43-91. See also Linear 
Systems; Pivoting strategies, 
for simultaneous linear 
équations 

Cramer’s rule and, 173 
Gaussian élimination method in, 
54-65 

augmented matrix in, 55-56 
définition of, 54-55 
elementary row operations in, 
56 

with partial pivoting strategies, 
65 

linear independence and, 71-78 
définitions of, 71-73 
theorems for, 74-75 
vectors and, 71-76 
linear Systems in, 43-50 
définitions of, 43-45 
Leontief closed models for, 
47-48 

Leontief Input-Output models 
for, 49-50 

matrix inversions and, 109-112 
pivoting strategies for, 65-71 
complété, 69-71 
partial, 65-67 
scaled, 67-68 
rank and, 78-84 

for column vectors, 78 
définition of, 78 
for row vectors, 78-82 
theorems for, 79, 82 
substitution methods in, 50-54 
first derived sets in, 50 
theory of solutions in, 84-88 
for consistent Systems, 86 
for homogenous Systems, 87 
nontrivial, 87 
trivial, 87 

Singular inversions, 93 
Skew symmetric matrices, 22 


Slack variables, 141 
Solutions of Systems, with linear 
differential équations, 
275-295 

with constant coefficients, 275-286 
quantity replacement in, 
277-278 

for general cases, 286-295 
définition of, 286 
theorems for, 287-294 
transitional matrices and, 
286-289 

Square matrices, 2 
inversions for, 102 
nonsingular, 115-116 
Strict inequalities, 129 
Submatrices, 29-32 
zéro, 31 

Substitution methods, in 

simultaneous linear 
équations, 50-54 
first derived sets in, 50 
Subtractions, of matrices, 7 
Symmetric matrices, 22 
inversions of, 113 
skew, 22 

T 

Taylor sériés, 216, 236 
Transitional matrices, 286-289 
Transpose matrices, 19-20 
commuted products of, 20 
définition of, 20 
inversions of, 113 
Trivial solutions, 87 

u 

Unions, within sets, 301 
Unit vectors, 34 

as inner product, 316 
Upper triangular matrices, 22 

eigenvalues/eigenvectors and, 191 
inversions of, 113 
LU décomposition in, 118 

v 

Vectors, 33-41. See also 

Eigenvectors; Orthonormal 
vectors 
column, 33 

linearly dépendent, 349 
rank for, 78 
work, 143 


components of, 33 
définitions, 33-34 
dimension of, 33 
eigenvectors, 177-212 
définitions of, 177-178 
eigenvalues and, 180-190 
linearly independent, 194-201 
nontrivial solutions with, 
185-186 

power methods with, 201-212 
properties of, 190-194 
force and, 40 
geometry of, 37-41 
angles in, 38f 
equivalency in, 40 
force effects on, 40 
measurement parameters in, 
39_40, 40f 

sum construction in, 38-39, 39f 
velocity effects on, 40 
inner products and, 315-344 
linear dependence and, 72 
linear independence and, 71-76 
magnitude of, 33 
nonzero, 316 
normalized, 316 
normalized, 34 
orthonormal, 320-327 

Gram-Schmidt process for, 
322-325,331-337 
projections for, 327-338 
QR décompositions for, 
334-344 
sets of, 320 

theorems for, 321-325 
row, 33 

as déterminant property, 
157-158 
rank for, 78-82 
unit, 34, 316 

as inner product, 316 

Velocity, vectors and, 40 

Vertical bar, in Simplex method, 142 

Vertices, 131 

Visualizations, for graphing 
inequalities, 128 

w 

Work column vectors, 143 

z 

Zéro matrices, 6 
nonzero v., 20 

Zéro submatrices, 31 



